The Free AI Leaderboard Everyone Trusts Has Quietly Become a $100M Business
A crowdsourced platform that ranks AI models through millions of head-to-head comparisons has turned its community-driven leaderboard into a fast-growing commercial evaluations business, hitting nine figures in annualized revenue less than a year after monetizing.
A platform many still think of as a free, open community project has turned into a serious revenue engine — proof that the unglamorous work of evaluating AI models can be just as lucrative as building them.
From Campus Research Project to Commercial Powerhouse
What started in 2023 as a research initiative out of UC Berkeley has grown into one of the most widely used tools for comparing AI model performance. The platform lets everyday users submit a prompt, see responses from two different AI models side by side, and vote on which answer is better. Over time, that simple mechanic generated more than ten million community evaluations, making the resulting leaderboard a trusted reference point across the AI industry.
For most of its existence, that leaderboard was the platform's entire identity — a free public good rather than a business. That changed last September, when the company introduced a paid evaluations product aimed at AI labs and enterprises wanting deeper, more structured performance analytics drawn from the same community of evaluators.
Revenue Growth That Caught Up Fast
The numbers tell a striking growth story. When the company announced a large funding round at the start of the year, its annualized revenue stood at roughly $30 million. Within months, that figure more than tripled to $100 million, underscoring just how much demand exists among AI developers for independent, large-scale model evaluation.
It's worth noting the company describes this figure as annualized recurring revenue, but its leadership has clarified that customers are actually billed based on consumption rather than fixed subscriptions — meaning the revenue, while substantial, isn't guaranteed to repeat in the same way traditional recurring revenue would.
Competing for the Same Dollar as Human-Labeling Giants
While the company doesn't have a direct rival running the same crowdsourced leaderboard model — a similar competitor shut down earlier this year — it does compete for budget with human-data-labeling companies that help AI labs fine-tune models after initial training. That broader category has seen enormous revenue growth recently, with rival firms reportedly approaching or surpassing $1 billion in annualized revenue themselves, reflecting just how much AI developers are willing to spend on refining and validating their models post-launch.
A tool built on volunteer goodwill can still become a genuine business — the trick is monetizing the insight without breaking the trust that made the community show up in the first place.
- $100M in annualized revenue. Reached just eight months after the company launched its paid evaluations service.
- Rapid acceleration. Revenue more than tripled from roughly $30 million at the start of the year.
- Consumption-based, not subscription. Leadership clarified the revenue is usage-driven rather than guaranteed recurring income.
- Backed by major investors. The company has raised $250 million total at a $1.7 billion valuation.
- Competing in a booming category. It vies for budget with human-data-labeling firms also seeing explosive revenue growth.