No reviews yetBe the first to leave a review for IndiaSocialBench
Maker
📌
Today, I'm very excited to launch something close to my heart:
Introducing, IndiaSocialBench. An eval built from scratch to measure how well a language model can understand Indian context. Whenever a model releases, there is always a talk about coding capabilities & general/emotional intelligence.
However, there is no eval to measure how good is that model with Indian context. The IndiaSocialBench is a honest attempt at that.
Built around 18 scenarios, across 8 social dimension totalling 50 datasets. Each model go through all the 50 datasets, and thus a detailed analysis will be made leading to final leader board.
Currently, started with 3 languages: English, Hindi & Hinglish. Will be expanded to other Indian languages gradually.
I love evals, and had a lot of fun building this. Very excited to open the public beta to everyone, and looking forward for feedback & suggestions.
