Abstract
Programming language and library choices are crucial to software reliabilityand security. Poor or inconsistent choices can lead to increased technicaldebt, security vulnerabilities, and even catastrophic failures insafety-critical systems. As Large Language Models (LLMs) play an increasingrole in code generation, it is essential to understand how they make thesedecisions. However, little is known about their preferences when selectingprogramming languages and libraries for different coding tasks. To fill thisgap, this study provides the first in-depth investigation into LLM preferencesfor programming languages and libraries used when generating code. We assessthe preferences of eight diverse LLMs by prompting them to complete variouscoding tasks, including widely-studied benchmarks and the more practical taskof generating the initial structural code for new projects (a crucial step thatoften determines a project's language or library choices). Our findings reveal that LLMs heavily favour Python when solvinglanguage-agnostic problems, using it in 90%-97% of cases for benchmark tasks.Even when generating initial project code where Python is not a suitablelanguage, it remains the most-used language in 58% of instances. Moreover, LLMscontradict their own language recommendations in 83% of project initialisationtasks, raising concerns about their reliability in guiding language selection.Similar biases toward well-established libraries further create seriousdiscoverability challenges for newer open-source projects. These resultshighlight the need to improve LLMs' adaptability to diverse programmingcontexts and to develop mechanisms for mitigating programming language andlibrary bias.