This project won 1000 EUR at AI4UX Hackathon 2021
Commentree (Evaluate your UX through actual users!)
The Problem
UX designers value user feedback highest. The user's word is the gold standard. But we often overlook the greatest source of user feedback: social media. In ignoring the 'collective knowledge' of their user group and by not being able to listen to their UI complaints online, UX designers miss out on important insights.
The Solution
Using machine learning for social media analysis, we mine existing user feedback (user reviews, tweets, etc) about a product available on social media sites like Twitter, Facebook, Instagram to recommend UX/UI design changes. This gives the UX designer continuous and direct feedback from their actual users, and our analysis recommends changes to the design.
The Framework
We use the below framework to measure our product's UX from online reviews posted to social media.
Implementing the Framework on Social Media User Feedback:
| UX Framework Attribute | Words to look out for in user feedback |
|---|---|
| Useful | Understandable, Consistent, Error, Approachable, ... |
| Desirable | Appropriate, Efficient, Truthful, Engaging, ... |
| Accessible | Disability, Visibility, Read Aloud, Difficult, ... |
| Secure | Permission, Location, Data, Confidence, Reliable, ... |
| Usability | Consistent, Completion, Informing, Navigation, ... |
Process
The Data
The users' feedback has been sourced from Twitter & App store reviews for this prototype, but can be sourced from any social media or app platform we want. The text data from the tweets contain words that can trigger one of the seven prongs of our UX framework. By mapping the text data of the tweet to our framework, we can evaluate the UX of the product.
The Analysis
We use multiple analysis techniques from natural language processing (NLP) and supervised machine learning to do analysis on user tweets. Some of these techniques include:
1. Sentiment analysis
Gain a measure of user satisfaction from the tweet by algorithmically measuring how positive or negative the sentiment of the tweet is.
2. n-gram analysis
Tokenization and frequency analysis of text data from tweets and user reviews.
3. Measuring UX success accroding to Framework
Use the 7-prong framework to determine the quality of the UX, from user feedback.
4. Aid strategy
Prioritze which design bugs to fix first by visualizing frequency of particular complaints in user feedback.
The Insights
Frequently Occuring Words
Negative Sentiment Distribution
Positive Sentiment Distribution
Word Cloud for representing most used words
Architecture Modules
1. Collect User Feedback from tweets
2. Analyze feedback to evaluate UX
3. Gain insights in the form of graphs
4. Suggest changes in UX according to user feedback
How to run: Files & What they do
Collect Tweets: Collected using calls to the Twitter API in listen_tweets.py. Sample data has also been provided.
Analysis: Run analysis on collected tweets using AI4UX.ipynb
Insight: Run visual analysis to gain insights into your users' experience by running AI4UX.ipynb
Tech Stack
- Jupyter notebook & Colab
- Data analysis: pandas, numpy
- Data visualization: seaboarn, matplotlib
- Natural Language Processing: nltk, textblob, wordcloud
- Twitter API
- MongoDB
- Adobe XD
The Team
- Daksha Dixit
- Simran Singh
- Arnav Ajay
- Arjun Bahuguna