Abstract
We introduce a new unsupervised text embedding method, Meta-Task Promptingwith Explicit One-Word Limitation (MetaEOL), for generating high-qualitysentence embeddings from Large Language Models (LLMs) without the need formodel fine-tuning. Leveraging meta-task prompting, MetaEOL guides LLMs toproduce embeddings through a series of carefully designed prompts that addressmultiple representational aspects. Our comprehensive experiments demonstratethat embeddings averaged from various meta-tasks are versatile embeddings thatyield competitive performance on Semantic Textual Similarity (STS) benchmarksand excel in downstream tasks, surpassing contrastive-trained models. Ourfindings suggest a new scaling law, offering a versatile and resource-efficientapproach for embedding generation across diverse scenarios.