Abstract
There are several distinct failure modes for overoptimization of systems onthe basis of metrics. This occurs when a metric which can be used to improve asystem is used to an extent that further optimization is ineffective orharmful, and is sometimes termed Goodhart's Law. This class of failure is oftenpoorly understood, partly because terminology for discussing them is ambiguous,and partly because discussion using this ambiguous terminology ignoresdistinctions between different failure modes of this general type. This paperexpands on an earlier discussion by Garrabrant, which notes there are "(atleast) four different mechanisms" that relate to Goodhart's Law. This paper isintended to explore these mechanisms further, and specify more clearly how theyoccur. This discussion should be helpful in better understanding these types offailures in economic regulation, in public policy, in machine learning, and inArtificial Intelligence alignment. The importance of Goodhart effects dependson the amount of power directed towards optimizing the proxy, and so theincreased optimization power offered by artificial intelligence makes itespecially critical for that field.