Source: The JetBrains Kotlin Blog — 2026-07-08
Summary
JetBrains released an open, SWE-bench-style benchmark for evaluating AI coding agents specifically on Kotlin repository-level tasks, publishing both the dataset and a public leaderboard.
Key Takeaways
- 105 tasks sourced from active open-source Kotlin repositories, each requiring interpreting a real issue, navigating repo context, and producing a patch verified in a containerized test harness.
- In the first published run, Claude Code with Opus 4.7 (xhigh) topped the leaderboard, resolving 90 of 105 tasks (85.71%).
- The leaderboard is public at kotlinlang.org/benchmark, with benchmark assets published on GitHub.
- JetBrains plans to expand coverage to Android and Kotlin Multiplatform and evaluate additional agent/model configurations, positioning it as an ongoing, language-specific complement to generic benchmarks like SWE-bench.