Robotic systems are more present in our society everyday. In human-robotenvironments, it is crucial that end-users may correctly understand theirrobotic team-partners, in order to collaboratively complete a task. To increaseaction understanding, users demand more explainability about the decisions bythe robot in particular situations. Recently, explainable robotic systems haveemerged as an alternative focused not only on completing a task satisfactorily,but also on justifying, in a human-like manner, the reasons that lead to makinga decision. In reinforcement learning scenarios, a great effort has beenfocused on providing explanations using data-driven approaches, particularlyfrom the visual input modality in deep learning-based systems. In this work, wefocus rather on the decision-making process of reinforcement learning agentsperforming a task in a robotic scenario. Experimental results are obtainedusing 3 different set-ups, namely, a deterministic navigation task, astochastic navigation task, and a continuous visual-based sorting object task.As a way to explain the goal-driven robot's actions, we use the probability ofsuccess computed by three different proposed approaches: memory-based,learning-based, and introspection-based. The difference between theseapproaches is the amount of memory required to compute or estimate theprobability of success as well as the kind of reinforcement learningrepresentation where they could be used. In this regard, we use thememory-based approach as a baseline since it is obtained directly from theagent's observations. When comparing the learning-based and theintrospection-based approaches to this baseline, both are found to be suitablealternatives to compute the probability of success, obtaining high levels ofsimilarity when compared using both the Pearson's correlation and the meansquared error.