SimKey: A Semantically Aware Key Module for Watermarking Language Models

  • 2025-11-03 18:20:37
  • Shingo Kodama, Haya Diwan, Lucas Rosenblatt, R. Teal Witter, Niv Cohen
  • 0

Abstract

The rapid spread of text generated by large language models (LLMs) makes itincreasingly difficult to distinguish authentic human writing from machineoutput. Watermarking offers a promising solution: model owners can embed animperceptible signal into generated text, marking its origin. Most leadingapproaches seed an LLM's next-token sampling with a pseudo-random key that canlater be recovered to identify the text as machine-generated, while onlyminimally altering the model's output distribution. However, these methodssuffer from two related issues: (i) watermarks are brittle to simplesurface-level edits such as paraphrasing or reordering; and (ii) adversariescan append unrelated, potentially harmful text that inherits the watermark,risking reputational damage to model owners. To address these issues, weintroduce SimKey, a semantic key module that strengthens watermark robustnessby tying key generation to the meaning of prior context. SimKey useslocality-sensitive hashing over semantic embeddings to ensure that paraphrasedtext yields the same watermark key, while unrelated or semantically shiftedtext produces a different one. Integrated with state-of-the-art watermarkingschemes, SimKey improves watermark robustness to paraphrasing and translationwhile preventing harmful content from false attribution, establishingsemantic-aware keying as a practical and extensible watermarking direction.

 

Quick Read (beta)

loading the full paper ...