Abstract
We propose a novel energy function for Dense Associative Memory (DenseAM)networks, the log-sum-ReLU (LSR), inspired by optimal kernel densityestimation. Unlike the common log-sum-exponential (LSE) function, LSR is basedon the Epanechnikov kernel and enables exact memory retrieval with exponentialcapacity without requiring exponential separation functions. Moreover, itintroduces abundant additional \emph{emergent} local minima while preservingperfect pattern recovery -- a characteristic previously unseen in DenseAMliterature. Empirical results show that LSR energy has significantly more localminima (memories) that have comparable log-likelihood to LSE-based models.Analysis of LSR's emergent memories on image datasets reveals a degree ofcreativity and novelty, hinting at this method's potential for both large-scalememory storage and generative tasks.