X-FACTR is a multilingual benchmark for probing factual knowledge in language models. Prompts in 23 languages are created by native speakers to probe factual knowledge in LMs by having them fill in the blanks of prompts such as "Punta Cana is located in _." We provide both the benchmark containing prompts and facts and the code to evaluate LMs. For more details, check out our paper X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models and our project page