Abstract
Large language models (LLMs) have demonstrated dominating performance in manyNLP tasks, especially on generative tasks. However, they often fall short insome information extraction tasks, particularly those requiring domain-specificknowledge, such as Biomedical Named Entity Recognition (NER). In this paper,inspired by Chain-of-thought, we leverage the LLM to solve the Biomedical NERstep-by-step: break down the NER task into entity span extraction and entitytype determination. Additionally, for entity type determination, we injectentity knowledge to address the problem that LLM's lack of domain knowledgewhen predicting entity category. Experimental results show a significantimprovement in our two-step BioNER approach compared to previous few-shot LLMbaseline. Additionally, the incorporation of external knowledge significantlyenhances entity category determination performance.