Abstract
Tracking entities throughout a procedure described in a text is challengingdue to the dynamic nature of the world described in the process. Firstly, wepropose to formulate this task as a question answering problem. This enables usto use pre-trained transformer-based language models on other QA benchmarks byadapting those to the procedural text understanding. Secondly, since thetransformer-based language models cannot encode the flow of events bythemselves, we propose a Time-Stamped Language Model~(TSLM model) to encodeevent information in LMs architecture by introducing the timestamp encoding.Our model evaluated on the Propara dataset shows improvements on the publishedstate-of-the-art results with a $3.1\%$ increase in F1 score. Moreover, ourmodel yields better results on the location prediction task on the NPN-Cookingdataset. This result indicates that our approach is effective for proceduraltext understanding in general.