Rumour stance classification, defined as classifying the stance of specificsocial media posts into one of supporting, denying, querying or commenting onan earlier post, is becoming of increasing interest to researchers. While mostprevious work has focused on using individual tweets as classifier inputs, herewe report on the performance of sequential classifiers that exploit thediscourse features inherent in social media interactions or 'conversationalthreads'. Testing the effectiveness of four sequential classifiers -- HawkesProcesses, Linear-Chain Conditional Random Fields (Linear CRF), Tree-StructuredConditional Random Fields (Tree CRF) and Long Short Term Memory networks (LSTM)-- on eight datasets associated with breaking news stories, and looking atdifferent types of local and contextual features, our work sheds new light onthe development of accurate stance classifiers. We show that sequentialclassifiers that exploit the use of discourse properties in social mediaconversations while using only local features, outperform non-sequentialclassifiers. Furthermore, we show that LSTM using a reduced set of features canoutperform the other sequential classifiers; this performance is consistentacross datasets and across types of stances. To conclude, our work alsoanalyses the different features under study, identifying those that best helpcharacterise and distinguish between stances, such as supporting tweets beingmore likely to be accompanied by evidence than denying tweets. We also setforth a number of directions for future research.