Abstract
We demonstrate a network visualization technique to analyze the recurrentstate inside the LSTMs/GRUs used commonly in language and acoustic models.Interpreting intermediate state and network activations inside end-to-endmodels remains an open challenge. Our method allows users to understand exactlyhow much and what history is encoded inside recurrent state in graphemesequence models. Our procedure trains multiple decoders that predict priorinput history. Compiling results from these decoders, a user can obtain asignature of the recurrent kernel that characterizes its memory behavior. Wedemonstrate this method's usefulness in revealing information divergence in thebases of recurrent factorized kernels, visualizing the character-leveldifferences between the memory of n-gram and recurrent language models, andextracting knowledge of history encoded in the layers of grapheme-basedend-to-end ASR networks.