Automated Literature Review Using NLP Techniques and LLM-Based Retrieval-Augmented Generation

  • 2024-11-27 18:27:07
  • Nurshat Fateh Ali, Md. Mahdi Mohtasim, Shakil Mosharrof, T. Gopi Krishna
  • 0

Abstract

This research presents and compares multiple approaches to automate thegeneration of literature reviews using several Natural Language Processing(NLP) techniques and retrieval-augmented generation (RAG) with a Large LanguageModel (LLM). The ever-increasing number of research articles provides a hugechallenge for manual literature review. It has resulted in an increased demandfor automation. Developing a system capable of automatically generating theliterature reviews from only the PDF files as input is the primary objective ofthis research work. The effectiveness of several Natural Language Processing(NLP) strategies, such as the frequency-based method (spaCy), the transformermodel (Simple T5), and retrieval-augmented generation (RAG) with Large LanguageModel (GPT-3.5-turbo), is evaluated to meet the primary objective. The SciTLDRdataset is chosen for this research experiment and three distinct techniquesare utilized to implement three different systems for auto-generating theliterature reviews. The ROUGE scores are used for the evaluation of all threesystems. Based on the evaluation, the Large Language Model GPT-3.5-turboachieved the highest ROUGE-1 score, 0.364. The transformer model comes insecond place and spaCy is at the last position. Finally, a graphical userinterface is created for the best system based on the large language model.

 

Quick Read (beta)

loading the full paper ...