Title:	An Empirical Study on LLM-Proposed Performance Improvements in Real-World Software
DNr:	Berzelius-2026-145
Project Type:	LiU Berzelius
Principal Investigator:	Philipp Leitner <philipp.leitner@chalmers.se>
Affiliation:	Chalmers tekniska högskola
Duration:	2026-04-23 – 2026-11-01
Classification:	10205
Keywords:

Abstract

Large Language Models (LLMs) can generate code, but can they generate fast code for complex, real-world software systems? In this study, we investigate this question using a dataset of 65 tasks mined from performance-critical open-source Java projects. Unlike prior studies, which focused on algorithmic puzzles, we conduct experiments on actual performance-sensitive production code and employ developer-written JMH benchmarks to rigorously validate performance gains against human baselines.