Pretrained language models have been suggested as a possible alternative orcomplement to structured knowledge bases. However, this emerging LM-as-KBparadigm has so far only been considered in a very limited setting, which onlyallows handling 21k entities whose single-token name is found in common LMvocabularies. Furthermore, the main benefit of this paradigm, namely queryingthe KB using a variety of natural language paraphrases, is underexplored sofar. Here, we formulate two basic requirements for treating LMs as KBs: (i) theability to store a large number facts involving a large number of entities and(ii) the ability to query stored facts. We explore three entity representationsthat allow LMs to represent millions of entities and present a detailed casestudy on paraphrased querying of world knowledge in LMs, thereby providing aproof-of-concept that language models can indeed serve as knowledge bases.