Abstract
Vision-and-Language Navigation (VLN) in continuous environments requiresagents to interpret natural language instructions while navigatingunconstrained 3D spaces. Existing VLN-CE frameworks rely on a two-stageapproach: a waypoint predictor to generate waypoints and a navigator to executemovements. However, current waypoint predictors struggle with spatialawareness, while navigators lack historical reasoning and backtrackingcapabilities, limiting adaptability. We propose a zero-shot VLN-CE frameworkintegrating an enhanced waypoint predictor with a Multi-modal Large LanguageModel (MLLM)-based navigator. Our predictor employs a stronger vision encoder,masked cross-attention fusion, and an occupancy-aware loss for better waypointquality. The navigator incorporates history-aware reasoning and adaptive pathplanning with backtracking, improving robustness. Experiments on R2R-CE andMP3D benchmarks show our method achieves state-of-the-art (SOTA) performance inzero-shot settings, demonstrating competitive results compared to fullysupervised methods. Real-world validation on Turtlebot 4 further highlights itsadaptability.