Abstract
Large Language Models (LLMs) have achieved remarkable progress in reasoning,alignment, and task-specific performance. However, ensuring harmlessness inthese systems remains a critical challenge, particularly in advanced modelslike DeepSeek-R1. This paper examines the limitations of Reinforcement Learning(RL) as the primary approach for reducing harmful outputs in DeepSeek-R1 andcompares it with Supervised Fine-Tuning (SFT). While RL improves reasoningcapabilities, it faces challenges such as reward hacking, generalizationfailures, language mixing, and high computational costs. We propose hybridtraining approaches combining RL and SFT to achieve robust harmlessnessreduction. Usage recommendations and future directions for deployingDeepSeek-R1 responsibly are also presented.