Watch Wallie react to Minecraft in real-time 🎙️ (open-source AI streamer)

@lucas_cook2 Thanks! That "feels alive" part is exactly what I was chasing

On latency, a few things stack up: • I don't run the LLM on every frame. A lightweight pHash change-detector + an "attention" engine decide when something's actually worth reacting to, so the heavy model only fires on real events (scene changes, getting jumped, entering a cave). That kills most of the wasted latency right there. • Everything streams. LLM tokens flow into a sentence splitter, and the FIRST sentence is sent to TTS before the full reply is even done — so audio starts ASAP instead of waiting on the whole response. • Sentences are pipelined: while sentence 1 is playing, sentence 2's TTS is already being synthesized in parallel. • TTS is tuned for first-audio (smaller server-side chunking), and frame prep (JPEG resize) runs off the event loop so it never stalls playback. Honest take: it's reaction-latency (~1-2s event→voice), not twitch-reflex — works great as a co-host reacting to what happens, not for frame-perfect input. It's all open source if you want to dig into the pipeline: https://github.com/Alradyin/wallie-V2

@madison_brooks2 Thanks! Built for it — a rolling summarizer keeps context bounded over long sessions, anti-repetition stops it looping, and auto-breaks + rate-limit backoff keep it paced for 24/7. The real ceiling is usually your LLM provider's rate limits, not the app.

@freya24 That's the dream There's already a cross-session memory layer (it remembers topics, regulars, running threads). But structured gameplay memory — "my base is here, my goal is diamonds, my build style" — is exactly the next step. Persistent goals + spatial memory = feels like a real player. Roadmap'd