Understanding Catastrophic Forgetting in Language Models via Implicit Inference
icl_vs_if/ contains instructions, code, and data for replicating our experiments on recovering the pretrained capability of in-context learning for instruction-tuned models.
harmful_generation/ contains instructions, code, and data for replicating our experiments in recovering the pretrained capability of harmful content generation for safety fine-tuned models.