PhysNLU: A Language Resource for Evaluating Natural Language Understanding and Explanation Coherence in Physics

  • 2022-01-12 02:32:40
  • Jordan Meadows, Zili Zhou, Andre Freitas
  • 1


In order for language models to aid physics research, they must first encoderepresentations of mathematical and natural language discourse which lead tocoherent explanations, with correct ordering and relevance of statements. Wepresent a collection of datasets developed to evaluate the performance oflanguage models in this regard, which measure capabilities with respect tosentence ordering, position, section prediction, and discourse coherence.Analysis of the data reveals equations and sub-disciplines which are mostcommon in physics discourse, as well as the sentence-level frequency ofequations and expressions. We present baselines which demonstrate howcontemporary language models are challenged by coherence related tasks inphysics, even when trained on mathematical natural language objectives.


Quick Read (beta)

loading the full paper ...