Launching today
LoRA Speedrun
A verified speedrun leaderboard for LLM fine-tuning
6 followers
A verified speedrun leaderboard for LLM fine-tuning
6 followers
Fine-tune the same model to the same score on the same GPU — fastest training run wins. Every record is re-run 3x with fresh seeds by a referee before it counts, so no self-reported numbers. The baseline took 11:57; three days and three verified records later, the community has it at 1:44. Attempting is free.
A public log of every submission attempt would be great, even just the timestamps and which configs were tried even if they failed verification. Right now you only see the records that stuck, but seeing the misses is honestly where half the learning happens for people trying to figure out what actually works.
One thing that would be great is seeing the actual hyperparameter differences between top runs, like what learning rate or batch size changes helped people cut their time down. Right now the leaderboard just shows the numbers but not what got them there, which makes it harder to iterate.
honestly love that the runs get verified by a referee, that takes out all the guesswork. one thing that would be cool is a public log of what configs each record used, like the actual hyperparameters and dataset choices, so others can try to beat them instead of just chasing a number