Reducing Reasoning Costs: The Path of Optimization for Chain of Thought via Sparse Attention Mechanism

  • 2025-01-23 16:09:32
  • Libo Wang
  • 0

Abstract

In order to address the chain of thought in the large language modelinference cost surge, this research proposes to use a sparse attentionmechanism that only focuses on a few relevant tokens. The researcherconstructed a new attention mechanism and used GiantRabbit trained with customGPTs as an experimental tool. The experiment tested and compared the reasoningtime, correctness score and chain of thought length of this model and o1Preview in solving the linear algebra test questions of MIT OpenCourseWare. Theresults show that GiantRabbit's reasoning time and chain of thought length aresignificantly lower than o1 Preview. It verifies the feasibility of sparseattention mechanism for optimizing chain of thought reasoning. Detailedarchitectural details and experimental process have been uploaded to Github,the link is:https://github.com/brucewang123456789/GeniusTrail.git.

 

Quick Read (beta)

loading the full paper ...