Abstract
Large language models (LLMs) have demonstrated impressive capabilities innatural language generation. However, their output quality can be inconsistent,posing challenges for generating natural language from logical forms (LFs).This task requires the generated outputs to embody the exact semantics of LFs,without missing any LF semantics or creating any hallucinations. In this work,we tackle this issue by proposing a novel generate-and-rerank approach. Ourapproach involves initially generating a set of candidate outputs by promptingan LLM and subsequently reranking them using a task-specific reranker model. Inaddition, we curate a manually collected dataset to evaluate the alignmentbetween different ranking metrics and human judgements. The chosen rankingmetrics are utilized to enhance the training and evaluation of the rerankermodel. By conducting extensive experiments on three diverse datasets, wedemonstrate that the candidates selected by our reranker outperform thoseselected by baseline methods in terms of semantic consistency and fluency, asmeasured by three comprehensive metrics. Our findings provide strong evidencefor the effectiveness of our approach in improving the quality of generatedoutputs.