We propose a Long Short-Term Memory (LSTM) with attention mechanism toclassify psychological stress from self-conducted interview transcriptions. Weapply distant supervision by automatically labeling tweets based on theirhashtag content, which complements and expands the size of our corpus. Thisadditional data is used to initialize the model parameters, and which it isfine-tuned using the interview data. This improves the model's robustness,especially by expanding the vocabulary size. The bidirectional LSTM model withattention is found to be the best model in terms of accuracy (74.1%) andf-score (74.3%). Furthermore, we show that distant supervision fine-tuningenhances the model's performance by 1.6% accuracy and 2.1% f-score. Theattention mechanism helps the model to select informative words.