Late Gadolinium Enhanced Cardiac MRI (LGE-CMRI) for detecting atrial scars inatrial fibrillation (AF) patients has recently emerged as a promising techniqueto stratify patients, guide ablation therapy and predict treatment success.Visualisation and quantification of scar tissues require a segmentation of boththe left atrium (LA) and the high intensity scar regions from LGE-CMRI images.These two segmentation tasks are challenging due to the cancelling of healthytissue signal, low signal-to-noise ratio and often limited image quality inthese patients. Most approaches require manual supervision and/or a secondbright-blood MRI acquisition for anatomical segmentation. Segmenting both theLA anatomy and the scar tissues automatically from a single LGE-CMRIacquisition is highly in demand. In this study, we proposed a novel fullyautomated multiview two-task (MVTT) recursive attention model working directlyon LGE-CMRI images that combines a sequential learning and a dilated residuallearning to segment the LA (including attached pulmonary veins) and delineatethe atrial scars simultaneously via an innovative attention model. Compared toother state-of-the-art methods, the proposed MVTT achieves compellingimprovement, enabling to generate a patient-specific anatomical and atrial scarassessment model.