Large Language Models and the Reverse Turing Test

  • 2022-07-28 22:22:47
  • Terrence Sejnowski
  • 17

Abstract

Large Language Models (LLMs) have been transformative. They are pre-trainedfoundational models that can be adapted with fine tuning to many differentnatural language tasks, each of which previously would have required a separatenetwork model. This is one step closer to the extraordinary versatility ofhuman language. GPT-3 and more recently LaMDA can carry on dialogs with humanson many topics after minimal priming with a few examples. However, there hasbeen a wide range of reactions on whether these LLMs understand what they aresaying or exhibit signs of intelligence. This high variance is exhibited inthree interviews with LLMs reaching wildly different conclusions. A newpossibility was uncovered that could explain this divergence. What appears tobe intelligence in LLMs may in fact be a mirror that reflects the intelligenceof the interviewer, a remarkable twist that could be considered a ReverseTuring Test. If so, then by studying interviews we may be learning more aboutthe intelligence and beliefs of the interviewer than the intelligence of theLLMs.

 

Quick Read (beta)

loading the full paper ...