This repository contains the code for the paper "Journey of ChatGPT from Prompts to Stories in Games: the Positive, the Negative, and the Neutral" accepted at IEEE ICCE-Berlin 2023.
This paper presents a study on biases present in the story endings for story-driven games generated by ChatGPT. The study uses various prompts to assess the biases in ChatGPT's output. The results emphasize a consistent inclination towards positive endings in the stories generated by ChatGPT. Even when explicitly instructed to generate neutral endings, ChatGPT exhibited a bias towards positive outcomes. These biases raise concerns regarding the training data and alignment processes used by OpenAI to train ChatGPT, as they may reflect societal biases or the preferences of the majority of the data. Addressing these biases is crucial to ensure that these models align with societal norms and avoid reinforcing existing biases. Future studies should concentrate on developing methods to reduce biases in AI language models and enhance the ethical perspective of these technologies.
main.py: The main script for data generation.requirements.txt: The requirements file for the project.data/: The directory containing the generated data.chatgpt_evaluation.py: The script for evaluating the generated data using ChatGPT.eval_analysis.py: The script for analyzing the evaluation results.prompt_runner.py: The script for running the data generation process.
- Create a virtual environment (if needed):
conda create -n chatgpt-story python=3.11and activate it:
conda activate chatgpt-story- Install the requirements:
pip install -r requirements.txt- Run the script for data generation:
python main.py