Abstract: In this paper, we study how LLMs like GPT store and retrieve facts to answer factual queries. Using a pretrained GPT-2 model, we evaluate factual accuracy on knowledge datasets, math ...