In the United States, 25% or greater than 200 billion dollars of hospitalspending accounts for administrative costs that involve services for medicalcoding and billing. With the increasing number of patient records, manualassignment of the codes performed is overwhelming, time-consuming anderror-prone, causing billing errors. Natural language processing can automatethe extraction of codes/labels from unstructured clinical notes, which can aidhuman coders to save time, increase productivity, and verify medical codingerrors. Our objective is to identify appropriate diagnosis and procedure codesfrom clinical notes by performing multi-label classification. We usedde-identified data of critical care patients from the MIMIC-III database andsubset the data to select the ten (top-10) and fifty (top-50) most commondiagnoses and procedures, which covers 47.45% and 74.12% of all admissionsrespectively. We implemented state-of-the-art Bidirectional EncoderRepresentations from Transformers (BERT) to fine-tune the language model on 80%of the data and validated on the remaining 20%. The model achieved an overallaccuracy of 87.08%, an F1 score of 85.82%, and an AUC of 91.76% for top-10codes. For the top-50 codes, our model achieved an overall accuracy of 93.76%,an F1 score of 92.24%, and AUC of 91%. When compared to previously publishedresearch, our model outperforms in predicting codes from the clinical text. Wediscuss approaches to generalize the knowledge discovery process of ourMIMIC-BERT to other clinical notes. This can help human coders to save time,prevent backlogs, and additional costs due to coding errors.