See Where You Read with Eye Gaze Tracking and Large Language Model

  • 2024-10-17 06:23:57
  • Sikai Yang, Gang Yan, Wan Du
  • 0

Abstract

Losing track of reading progress during line switching can be frustrating.Eye gaze tracking technology offers a potential solution by highlighting readparagraphs, aiding users in avoiding wrong line switches. However, the gapbetween gaze tracking accuracy (2-3 cm) and text line spacing (3-5 mm) makesdirect application impractical. Existing methods leverage the linear readingpattern but fail during jump reading. This paper presents a reading trackingand highlighting system that supports both linear and jump reading. Based onexperimental insights from the gaze nature study of 16 users, two gaze errormodels are designed to enable both jump reading detection and relocation. Thesystem further leverages the large language model's contextual perceptioncapability in aiding reading tracking. A reading tracking domain-specificline-gaze alignment opportunity is also exploited to enable dynamic andfrequent calibration of the gaze results. Controlled experiments demonstratereliable linear reading tracking, as well as 84% accuracy in tracking jumpreading. Furthermore, real field tests with 18 volunteers demonstrated thesystem's effectiveness in tracking and highlighting read paragraphs, improvingreading efficiency, and enhancing user experience.

 

Quick Read (beta)

loading the full paper ...