Gemini Diffusion: Exploring faster, more thoughtful AI

Hi everyone!

I've been really interested in text generation models that use diffusion, since it's such a different approach from the usual autoregressive ones. After seeing models like Mercury and the Dream series, I thought it might take a while for the big players to release their own text diffusion models, so Google dropping Gemini Diffusion now is quite a surprise.

Gemini Diffusion is Google's new experimental model that generates text or code by refining noise step-by-step, much like image diffusion models. They're reporting it's significantly faster than their previous fastest models and produces more coherent text because it generates blocks of tokens at once.

What I find particularly compelling about the diffusion approach for text is the potential for models to generate content with more 'fully formed' thoughts. While non-streaming output might seem different initially, the possibility of more coherent and deeply considered responses is really attractive. The speed of Gemini Diffusion is really fast – 1479 token/s🤯this could indeed become a new paradigm for text generation.