From Logic to Language: A Trust Index for Problem Solving with LLMs

  • 2025-07-21 19:50:45
  • Tehseen Rug, Felix Böhmer, Tessa Pfattheicher
  • 0

Abstract

Classical computation, grounded in formal, logical systems, has been theengine of technological progress for decades, excelling at problems that can bedescribed with unambiguous rules. This paradigm, however, leaves a vast oceanof human problems -- those characterized by ambiguity, dynamic environments,and subjective context -- largely untouched. The advent of Large LanguageModels (LLMs) represents a fundamental shift, enabling computational systems toengage with this previously inaccessible domain using natural language. Thispaper introduces a unified framework to understand and contrast theseproblem-solving paradigms. We define and delineate the problem spacesaddressable by formal languages versus natural language. While solutions to theformer problem class can be evaluated using binary quality measures, the latterrequires a much more nuanced definition of approximate solution space takinginto account the vagueness, subjectivity and ambiguity inherent to naturallanguage. We therefore introduce a vector-valued trust index Q, which reflectssolution quality and distinguishes the binary correctness of formal solutionsfrom the continuous adequacy spectrum characteristic of natural languagesolutions. Within this framework, we propose two statistical qualitydimensions. Normalized bi-semantic entropy measures robustness and conceptualdiversity of LLM answers given semantic variation in problem formulations.Emotional valence maps subjective valuation of a solution to a quantifiablemetric that can be maximized by invoking statistical measures. The conceptsintroduced in this work will provide a more rigorous understanding of thecapabilities, limitations, and inherent nature of problem-solving in the age ofLLMs.

 

Quick Read (beta)

loading the full paper ...