A dataset for task-based recommendation conversation.
We provide all the data files under the V2 folder.
The conversation dataset is divided into several parts according to the task. Each conversation is stored in a .json file under the corresponding task folder.
The format of the .json file is:
{
"task": "The name of the task for this conversation.",
"preference_id": "The ID of the user profile used in this conversation.",
"task_context_id": "The ID of the task description used in this conversation.",
"preference": "Preference used in this conversation.",
"preference_zh": "Preference used in this conversation translated in Chinese.",
"task_context": "The scenario-specific task description used in this conversation",
"task_context_zh": "The scenario-specific task description used in this conversation translated in Chinese.",
"history": [
{
"role": "user",
"content": "Some text generated by user simulator LLM.",
"content_zh": "Some text generated by user simulator LLM translated in Chinese.",
"intent": "The intent annotation of the user simulator LLM.",
"intent_zh": "The intent annotation of the user simulator LLM translated in Chinese.",
},
{
"role": "assistant",
"content": "Some text generated by assistant LLM.",
"content_zh": "Some text generated by assistant LLM translated in Chinese.",
"hallucination": {
"hallucination": "Whether the assistant hallucinates in this turn.",
"memo": "The memo of the hallucination.",
},
},
// ...
],
"conflict": false,
"preference_summary": "The summary of the user simulator's preference extracted from the conversation.",
"rating": {
"Preference Alignment": "The rating of preference alignment.",
"Additional Preferences": "The rating of additional preferences.",
"Role-Playing Completeness": "The rating of role-playing completeness.",
"Memorization": "The rating of memorization.",
"Detail Level": "The rating of detail level.",
"Practical Usefulness": "The rating of practical usefulness.",
"Diversity": "The rating of diversity.",
},
}We provide the code for the framework RecUserSim in the RecUserSim folder.
Check the readme.md file in the RecUserSim folder for more details.