Robustness to noise is of utmost importance in reinforcement learningsystems, particularly in military contexts where high stakes and uncertainenvironments prevail. Noise and uncertainty are inherent features of militaryoperations, arising from factors such as incomplete information, adversarialactions, or unpredictable battlefield conditions. In RL, noise can criticallyimpact decision-making, mission success, and the safety of personnel. Rewardmachines offer a powerful tool to express complex reward structures in RLtasks, enabling the design of tailored reinforcement signals that align withmission objectives. This paper considers the problem of the robustness ofintelligence-driven reinforcement learning based on reward machines. Thepreliminary results presented suggest the need for further research inevidential reasoning and learning to harden current state-of-the-artreinforcement learning approaches before being mission-critical-ready.