When evaluating an answer choice for Reading Comprehension task, other answerchoices available for the question and the answers of related questions aboutthe same paragraph often provide valuable information. In this paper, wepropose a method to leverage the natural language relations between the answerchoices, such as entailment and contradiction, to improve the performance ofmachine comprehension. We use a stand-alone question answering (QA) system toperform QA task and a Natural Language Inference (NLI) system to identify therelations between the choice pairs. Then we perform inference using an IntegerLinear Programming (ILP)-based relational framework to re-evaluate thedecisions made by the standalone QA system in light of the relations identifiedby the NLI system. We also propose a multitask learning model that learns boththe tasks jointly.