Abstract
Leave-One-Out (LOO) provides an intuitive measure of feature importance butis computationally prohibitive. While Layer-Wise Relevance Propagation (LRP)offers a potentially efficient alternative, its axiomatic soundness in modernTransformers remains largely under-examined. In this work, we first show thatthe bilinear propagation rules used in recent advances of AttnLRP violate theimplementation invariance axiom. We prove this analytically and confirm itempirically in linear attention layers. Second, we also revisit CP-LRP as adiagnostic baseline and find that bypassing relevance propagation through thesoftmax layer -- backpropagating relevance only through the value matrices --significantly improves alignment with LOO, particularly in middle-to-lateTransformer layers. Overall, our results suggest that (i) bilinearfactorization sensitivity and (ii) softmax propagation error potentiallyjointly undermine LRP's ability to approximate LOO in Transformers.