This is the official code and data for our AAAI 2024 paper, "OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples"
Current LLM-generated text detectors lack robustness against attacks: they degrade detection accuracy by simply paraphrasing LLM-generated texts. Furthermore, there is the unexplored risk where malicious users might exploit LLMs to create texts specifically designed to evade detection.
In this paper, we propose OUTFOX, a framework that improves the robustness of LLM detectors by allowing both the detector and the attacker to consider each other's output. In this framework, the attacker uses the detector's prediction labels as examples for in-context learning and adversarially generates essays that are harder to detect, while the detector uses the adversarially generated essays as examples for in-context learning to learn to detect essays from a strong attacker.
Experiments in the domain of student essays show that...
- The proposed detector improves the detection performance on the attacker-generated texts by up to +41.3 points F1-score.
- The proposed detector shows a state-of-the-art detection performance: up to 96.9 points F1-score, beating existing detectors on non-attacked texts.
- The proposed attacker drastically degrades the performance of detectors by up to -57.0 points F1-score, massively outperforming the baseline paraphrasing method for evading detection.
- Feb 2024: Presented in AAAI 2024, Vancouver! [poster]
- Aug 2023: Our code and essay dataset are now available!
- python==3.9.2
$ python -m venv env
$ source env/bin/activate
$ pip install -r requirements.txt
Run any script in the scripts directory.
scripts/detection.sh is a script for our OUTFOX detection and baseline detections.
scripts/attacking.sh is a script for our OUTFOX attacking and baseline paraphrasing attack.
We created our dataset based on Kaggle FeedBack Prize, and our dataset contains 15,400 triplets of essay problem statements, human(native-student)-written essays, and LLM-generated essays. The native students range from 6th to 12th grade in the U.S.
We instruct three LMs to generate essays: ChatGPT(gpt-3.5-turbo-0613), GPT-3.5(text-davinci-003), and FLAN-T5-XXL.
We split the dataset into three parts: train/validation/test with 14400/500/500 examples, respectively.
This is supplemental information about the file names consisting of our dataset.
| File name | Content |
|---|---|
(train|valid|test)_problem_statements.pkl |
Essay problem statements in each set. |
(train|valid|test)_humans.pkl |
Human-written essays in each set. |
(train|valid|test)_lms.pkl |
LLM-generated essays in each set. |
Additionally, (train\|valid\|test)_contexts.pkl includes the prompts used to generate essays in each set. We use these to compute the likelihood in statistical outlier detectors.
We also provide the attacked essays by our OUTFOX attacker in data/chatgpt/test/test_outfox_attacks.pkl and the attacked essays by DIPPER in data/dipper/(chatgpt|text_davinci_003|flan_t5_xxl)/test_attacks.pkl.
If you find our code/data/models or ideas useful in your research, please consider citing our work as follows:
@InProceedings{Koike:OUTFOX:2024,
author = {Ryuto Koike and Masahiro Kaneko and Naoaki Okazaki},
title = {OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples},
booktitle = {Proceedings of the 38th AAAI Conference on Artificial Intelligence},
year = {2024},
month = {February},
address = {Vancouver, Canada}
}