A Multi-App Respond Tool
DPX-designer opened this issue · comments
A Multi-App Respond Tool
The Respond Tool is a web app designed for phones which allows our amazing volunteers to efficiently respond to low scoring product reviews on the Google Play store.
How it works
- A person (let's call them Sharon) gives a 2 star review of Firefox Android on Google Play, Sharon's review mentions that they had a problem with bookmarks.
- Respond Tool receives the review and adds it to a "response queue".
- A volunteer uses the Respond Tool to see if any new reviews have entered the queue, they see Sharon's review and open it.
- The volunteer writes a response to Sharon's review, providing some helpful feedback on how Sharon can resolve their issue.
- Once finished, the volunteer submits their response to Sharon.
- The volunteer's response is added to a second queue, a"moderation queue", ready for another volunteer to check and approve the response provided.
- Once three additional volunteers have approved of the response the response is then published to Google Play for Sharon to see.
The Respond Tool today
Today the Respond Tool allows volunteers to respond to low scoring reviews for Firefox Android on Google Play. No other store or product is supported.
The task
We wish to investigate how the Respond Tool may support:
- Multiple Mozilla products within the Google Play store or other app store.
- Multiple Stores and/or websites which allow comments and reviews of Mozilla products.
What is the journey the volunteer should take in order to respond to a low scoring review of product X on app store Y, or maybe it was a negative comment on social media site Z?
To provide a bit of scope, the start and finish points of your solutions should be the following:
Start: The volunteer opens the Respond Tool
Finish: The Volunteer clicks Submit, to send their written response to the moderation queue
Here are a few questions you may wish to consider:
- What are the screens the volunteer will see in each step?
- What buttons will they tap on in order to progress and eventually submit their response to the review/comment?
- How does the volunteer discover and navigate between reviews on different stores for different products?
- How could a volunteer be informed that one of their written responses failed moderation? A log? An inbox? Multiple inboxes, one for each app or store?
The Format
You can supply your designs in the following formats:
- Experience flow diagrams
- Sketch work
- Wireframes
This is not a visual design exercise but an experience flow exercise.
We create experience flow diagrams to map out in a lightweight fashion what screens or steps a person may encounter in order to fulfil a goal. Despite taking the form of simple arrows and boxes, experience flow diagrams form a crucial component of a well thought UX strategy.
Feel free to ask any questions here or on the Open Design Matrix room
We look forward to seeing what you come up with!
Deadline:
Monday 20th April
Tag:
[Ideas needed, User Experience]
A prototype for the Multi-App Respond Tool; below is the link to have a look at the design.
https://xd.adobe.com/view/fc27e511-b4fd-41d0-464d-b9f1c4d2f348-8a16/
Hi @devanshgajjar Thank you so much for this submission along with some super UI work!
I have a few questions regarding your user flow :)
Could you explain to me the "User sends query" step? Is the 'user' the volunteer using the respond tool or the person who has posted the review or comment on the app store/social media?
On the image below I have posed a question, assuming our volunteers can respond to a multitude of reviews from various websites and app stores, how does our volunteer discover/locate/browse the app or store they wish to begin responding too?
Moving onto your accompanying visuals, this exercise is not about visual design however with such an astonishing amount of effort put into creating a polished prototype I'll absolutely voice a few thoughts!
Visually your mock ups give me a Common Voice vibe, whether accidental or intentional I appreciate the nod to an existing Mozilla project.
What is your reasoning for breaking up the three moderation questions into 3 separate steps?
Interesting that you move the "Top contributions" section from the Home page to the volunteers profile. Similar feedback was actually voiced to me earlier this year and I'm curious what your reasoning is for doing the same?
Your strict colour scheme of red, blue and greyscale works very well and would no doubt help volunteers find their way and interpret their stats and progress easily.
All in all this is terrific, thank you very much! 👏
Hi again @mcroud,
By the user sends query I do indeed mean someone posting a review on the app.
For the second part of the query which refers to how to specify the query. We can add filter options like dropdowns to, Moderate or Respond to whatever app or app-store they like to.
I intentionally put visuals to show how this Multi-App Respond tool's model is astonishingly similar to Common Voice's model meaning, Common Voice has different languages and this tool has different apps & app store, also similar to Speak and Listen, this tool has to Respond and Moderate.
Now making the visual similar did not only make this exercise of mine a visual design but it did work on the Experiential part of it because you knowing that Common Voice exists, I am pretty sure you did not find it hard to figure out what is happening in the tool. So, in this way I was planning on bringing uniformity to both the campaigns so that mutual as well as new users respond and learn the respective tools easily irrespective fo the tool the come from. So the Common Voice vibe was indeed intentional and is needed in my opinion to bring uniformity-clarity to the users.
Talking about why I moved the "Top contributors" to the home page because I was trying to empathize the users, I have learned something that for someone to use our tool/product/service we have to offer something, anything small/rewarding while we are asking for something which "Top contributors" was satisfying; saying if you contribute well enough you might as well land on the list. Just trying to give something as a first impression while we are being helped. Also, they are not as personal as the things in volunteers profile are. We should not add things which are completely personal to the volunteer's profile.
Now, I would like to explain why I have put the moderations into three parts; it so happens while we are filling a form we have to rate something. The first question's answer is not biased at all, but when the second question comes, irrespective of the relation of the previous question we do tend to incline our answer considering the previous question's answer, so by blinding them from the previous answer the user may to some extent give a more clearer and unbiased review. The last step was just for confirmation.
N.B.: This is an aspect of what is called Gamification:
Talking about why I moved the "Top contributors" to the home page because I was trying to empathize the users, I have learned something that for someone to use our tool/product/service we have to offer something, anything small/rewarding while we are asking for something which "Top contributors" was satisfying; saying if you contribute well enough you might as well land on the list.
If you read about Bartle's Player Types you might want to consider adding some social aspects as well.
N.B.: This is an aspect of what is called Gamification:
If you read about Bartle's Player Types you might want to consider adding some social aspects as well
Hey @Ryuno-Ki , I do know a bit about gamification and thank you for sending me the article, it differentiated my thinking of gamification.
Talking about adding gamification in this tool is a good idea and for the matter of fact, I had thought of putting gamification concepts like streaks, badges, etc for the contributors, to motivate them, but I did not add it to the current version because doing the needed changes in the current model should be our priority and then we can start putting new models/aspects to the tool. In doing so the developers won't feel burdened and by the time users get used to the new interface we can give iterative updates. All in all, adding more gamification concepts will motive all the 'player types (from the article https://www.interaction-design.org/literature/article/bartle-s-player-types-for-gamification)' to contribute. Brilliant idea @Ryuno-Ki !
I suggested on Telegram/Matrix to provide some more engineering-related diagrams and got positive feedback.
So let's start with a sequence diagram:
The image was generated using mermaid.js. If you want to tinker with it, open the live editor and paste the code if it doesn't show up:
sequenceDiagram
participant Sharon
participant GooglePlayStore
participant RespondTool
participant Volunteer
participant Moderator
Sharon->>GooglePlayStore: Submit review with description of problem
GooglePlayStore->>RespondTool: Notifies
Note right of RespondTool: Added to "response queue"
RespondTool-->>Volunteer: Fetches review from "response queue"
Volunteer-->>RespondTool: Writes feedback on review
Note right of RespondTool: Added to "moderation queue"
RespondTool-->>Moderator: Check feedback
Moderator-->>RespondTool: Approve feedback
Note right of RespondTool: After three approvals
RespondTool->>GooglePlayStore: Publish feedback
GooglePlayStore-->>Sharon: See feedback
(Note, that I'm self-taught in regards of these types of diagrams. Feedback appreciated)
Interesting for this issue is the interplay between RespondTool with Volunteer and Moderator (sadly, without any name in the initial description :-( )
Okay, so let's take a closer look at the response tool:
Again, this flowchart was created with mermaid.js using:
graph TD
initial(Review in Response Queue)
volunteer[Write feedback to review]
moderation{Is feedback acceptable?}
rejected((Feedback is rejected))
approving[3 Moderators approve feedback]
approved[Feedback was approved]
submitted((Feedback is submitted to AppStore))
initial-->volunteer
volunteer-->moderation
moderation-- no -->rejected
moderation-- yes -->approving
approving-->approved
approved-->submitted
In case you need help interpreting it: Rounded corners denote an initial state. Square corners a process step. Diamonds are used for decisions. Circles denote final states.
From here, I feel some parts are missing …
- How long can a review stay in the response queue? (Can it outdate?)
- What happens, if several volunteers are writing a feedback? (Can a review be „locked” to a volunteer?)
- What happens, if a feedback does not earn enough approvals? (I assumed a rejection. Will the author of the feedback be informed? Can s/he/they edit it and resubmit? Will that be a new entry in the moderation queue? Can the old version still be looked at?)
- How can a volunteer become a moderator? Is there a Karma-system ala Stack Overflow? MozReps/employees? Something along the lines of „code owners” in Firefox or „council members” (?) of MDN?
This flow diagram will likely need additional iterations.
Okay, this one is likely really technical. It's about a class diagram to think about what properties and methods different objects have.
That is, what characterises them, what can they do and how do they relate to each other.
Feel free to tinker with it:
classDiagram
Person <|-- Author
Person <|-- Moderator
Person <|-- Volunteer
class App
App : String appId
App : AppStore appStore
class AppStore{
<<Enumeration>>
GooglePlay
iTunes
}
class Person
Person : String name
class Author
Author : +writes(Review review)
class Moderator
Moderator : List~Language~ speaks
Moderator : +approves(Feedback feedback)
class Volunteer
Volunteer : List~Language~ speaks
class Language
Language : String name
class Feedback
Feedback : Volunteer author
Feedback : String message
Feedback : Review inResponseTo
Feedback : List~Moderator~ approvedBy
class Review
Review : Author author
Review : String message
Review : Language writtenIn
Review : App app
Feedback "1" --> "1" Author
Feedback "1" --> "1" App
Review "1" --> "1" Volunteer
Review "1" --> "0..3" Moderator
Review "1" --> "1" Feedback
Volunteer "*" --> "1..*" Language
Moderator "*" --> "1..*" Language
Review "*" --> "1" Language
Feedback "*" --> "1" Language
App "*" --> "1" AppStore
I'm not done with this one (and likely confused the sites of relationship) but have to go to bed now. Take it with a grain of salt (or as starting point to dive into deeper details).
This diagram could be used by engineers for modelling. That is, defining classes, database tables and so on. Please don't hesitate to ask, if something is unclear / unknown.
Two more thoughts which came to me last night:
- What happens with what data if a volunteer/moderator deletes her/his/their account?
- In which context is the app used by a volunteer/moderator? (Keyword Empathy Map, Micromoments)
Shall I elaborate?
Hello @Ryuno-Ki
Apologies for the delay in feedback. Thank you so much for this volume of exploration! This is top work.
Looking at your Sequence diagram, I really like how it provides an overview of interactions between the key parties of the experience and the type of interactions which occur between them.
I was happy to see your class diagram highlight the factor of language as a key component, helping non-English responses receive the moderations they need is something we're keen to action.
I'm very interested in your flowchart. It clearly presents the process of a volunteer responding to a review and receiving moderations prior to the review being published.
What would be cool to see on such a diagram is how the volunteer selects the review to respond to. What menu or screen do they select from? How do they decide to respond to a review on Google Play or a mention of "Firefox" on Twitter? (The "Multi" app will support multiple app stores and websites. Not just reviews but possibly also comments and posts).
Thank you so much! 👏
I saw the above inputs and I must say they are so detailed and amazing! Although I feel the questions that are still left unanswered are: How will the volunteers interact with a respond tool that supports multiple Mozilla apps and reviews from multiple app stores and other sources.
Firstly, mentioning about the Moderate option. It will be the same with just the addition of details of - App Used, Source of the Review [optional]. I saw an option of Canned Responses, so the Canned Responses would be set according to the App Used i.e. the app that was reviewed by the user.
For the Respond option, I think it's better done than said, here are my low-fidelity wireframes explaining the same. I have made two versions
[The design might vary depending on the number of Mozilla apps and app stores]
Version 1 - 
Version 2 - 
2 is better than 1 in terms of easily navigating between different apps.
1 is better than 2 for a simplified version and is slightly less complicated for users to interact with.
Hm, perhaps a criterium could be that the volunteer should have prior experience with an app before she/he/they can write a response to a review?
Regarding the Wireframes: What is strictly necessary? I could imagine, that there is one screen which allows to pick App Store, the next one the pending reviews (awaiting reply) for apps in that App Store, the third one showing the review, the name of the app and the app store and a textfield for the response (is markup allowed, i.e. a WYSIWYG editor here?).
This way, every view is only delivering the context necessarily to progress through the form.
Instead of news, it could also be progressively disclosed.
Can't promise that I'll find time to refine the above diagrams. But I'm happy, that they sparked more conversations :-)
@nitin10s Wonderful! You've addressed the "multi-app" task very well and done so with these lovely wireframes, thank you!
I have an additional question for you 😀 Imagine that the volunteer using the Respond tool is able to access an 'inbox' or message centre to read their feedback messages from other volunteers. How would our volunteer access their messages relating to a particular app?
Would the volunteer need to select the app first from the main menu (similar to your 'version 1' app select screen) and then access an inbox - so a separate inbox for each app?
Would the volunteer have a single inbox in which they can filter for a specific app?
Thank you so much! 👏
@Ryuno-Ki Necessary items for a volunteer responding would likely be:
- An interface to select Responding and Moderating
- An interface(s) to locate/browse and select a review to respond to
- An interface(s) to respond to the selected review
Reviews currently support plain text only.
I'll rather name it Feedbacks/Reviews (Inbox sounds confusing unless there are other types of notifications in that inbox). The option should ideally be placed on the home screen rather than the Profile section for volunteers to use.
And yes it's best to have a filter option rather than separate inbox for separate apps. Something like this... [Even tweaked the current design for inbox]
__
Thank you @nitin10s these are super wireframes that tell the story very well 👍
I like how your proposed filter menu goes beyond a simple drop down, below was a proposal I had for an earlier prototype and I think they have a similar vibe 😄
@mcroud Yes, indeed they are quite similar :D
Hi, I've gone through all previous discussions and found them very useful.
I myself worked on the user flows and wireframes.
Detailed User flow :
I've tried to translate the user flow in a way to reduce the number of clicks required to respond to a review.
Proposed Wireframe :
Home screen -
Switching mode to moderate -
To moderate a response -
As @Rupali2412 also mentioned, the best way is indeed giving users an option to filter the reviews by the app in Respond and Moderate section.
Although, there is a small error in the wireframe "To moderate a response", you've used radio buttons for the three parameters that can be confusing for volunteers. We should either use checkboxes or stick to the current design. Views?
@nitin10s Yes, you are right. Use of radio buttons is not proper here. It looks like a predefined set of mutually exclusive options. I guess the current design is better as even beginner volunteers can understand the context easily. Thanks for identifying.
@Rupali2412 Wonderful! Thank you for such a clear user flow and accompanying wireframes.
I like how you've condensed the amount of steps to begin responding and moderating within your wireframes, good job.
I have a question. Looking at your flow diagram, we're assuming the user would likely select the platform before selecting the mode (respond/moderate).
Imagine that instead of having the app, platform and mode selection on a single page (as per your wireframes), you had them as separate steps - so a screen for each step. Would you ever consider having the volunteer select a platform/app first, then ask them whether they wish to respond or moderate second?
Thanks again!
@mcroud Thank you so much for the appreciation.
I would suggest the following flow:
Select app(products) >> select mode ( respond/moderate ) >> Select platform
I propose this flow as the volunteer might not be aware of multiple Mozilla apps(products). So by selecting an app volunteer is setting it as a default for next(future) response and moderations also.
Platform selection is at the end of the flow as the number of reviews or responses(to moderate) of in a platform can be displayed which may stimulate a user to select.
This is actually a good debatable topic. Whether to consider app selection or mode selection as the primary function.
- If a user wants to often switch between apps while moderating/responding
- The user wants to moderate/respond for a particular app only.
A huge well done and thank you to @devanshgajjar, @Ryuno-Ki, @nitin10s and @Rupali2412 for contributing and sharing such an amazing effort of work.
🎉 🎉 🎉
I have thoroughly enjoyed looking at all of your experience flow diagrams, wireframes and design work.
I have mentioned this thread during a few organisational meetings to enthuse about the power of open design, thank you for making it such a success. Later this year I will be constructing a new multi-app prototype and I look forward to sharing the results of our collaborated multi-app experience with the community.
Thank you again!
Hi @mcroud ! I know i'm way past the deadline. But i found the idea of a respond tool really cool and the discussion over here was really interesting. May i know if i can still contribute to this issue?
Thanks!
Hi @sunidhi-kashyap
The deadline for this project was fairly arbitrary, as this issue is still open I couldn't find any reason to argue against anyone who wishes to volunteer ideas and suggestions to this topic. So please feel free to submit a contribution to the brief, I look forward to seeing what you come up with!
Thank you @mcroud for giving me the chance.
After I read the issue, I thought of designing a simple and intuitive interface, which will get our task done with ease.
Below are some low-fi wire-frames I sketched, with explainations:
After the user opens the app, he will see the home-screen and will have 4 options. The flow-charts explain the flow per option on the menu:
The "almost there" wire-frames to give a visual of the Response Queue, and how the filter works:
I saw that there were way too many things going on with the potential respond tool. But every functionality you mentioned was important to be present in the web-app, for it to be a good tool. But getting everything done, might end up making the interface too congested and the user gets confused and reluctant to use the app. So, I divided the tasks into 4 main items and designed a "Bottom Menu Bar", to make the navigation easy.
I used a separate screen as a home page, which shows the volunteer the her/his latest submissions and allows to edit.
Also a separate screen to view the profile, which shows the notifications(mainly accepted and rejected submissions), which was mentioned by you in the issue.
I really enjoyed reading everyone's contributions!
@Ryuno-Ki Mentioned about gamification. I do believe it will encourage volunteers to do a better job, when rewarded. I wasn't able to add this into the interface, but i believe it is a really good feature.
I really hope i was of some help in this. I really feel that a response tool is a much needed product in today's time. It can help us improve the products for users and is a really positive approach towards satisfying a user's needs.
Thankyou.
I love your handwriting! It's beautiful :-)
I love your handwriting! It's beautiful :-)
Thank you :)
@sunidhi-kashyap These sketches, flows and wireframes are wonderful. Thank you so much for taking the time to produce these!
I really like the idea of a global respond and moderate queue, taking volunteers straight to the list and letting the filters do the work is a great take. How would you cater for volunteers who will only ever wish to deal with a particular platform? Perhaps the applied filters can be saved?
The Introduction of a bottom bar/global navigation fell inline with a prototype I made earlier in the year, It’s always nice to see another designer present a similar solution.
Including “Respond” and “Moderate” as key icons within that bar is a good suggestion, I suppose some research would help indicate the frequency that a volunteer would swap modes during a session.
On the subject of gamification, did you have any specific thoughts?
Once again, thank you so much for this top work 👏
Hi @mcroud,
I noticed, that you added a number of unread notifications as pillar to the footer.
Are you aware that those badge notifications can cause stress? (Picked one among several articles on that matter).
If you deem them necessary, please don't use red. Either a grayscale or the option to disable them would be recommended.
@Ryuno-Ki Fascinating, no I didn't! What a terrific article, thank you for bringing it to my attention.
I'm always eager to not contribute to any more digital annoyances in people's lives so I'm happy to act on this immediately, below shows a revision of these notification badges for the desktop styling I'm currently working on:
Top button is the previous version, below is the new, neutral version:
Thanks again!
@mcroud Thank you so much for the feedback, it was very encouraging!
"How would you cater for volunteers who will only ever wish to deal with a particular platform? Perhaps the applied filters can be saved?" -- Yeah, I didn't think about it before. I think we should focus on volunteers who would like to stick with a particular platform, and your idea of saving the applied filters is perfect! I mean, we wouldn't have to add an extra screen, and we can provide the user to edit the filters too!
"The Introduction of a bottom bar/global navigation fell inline with a prototype I made earlier in the year, It’s always nice to see another designer present a similar solution." -- Thanks! It was very nice of you to mention how my approach matched your design thinking here.
"I suppose some research would help indicate the frequency that a volunteer would swap modes during a session." -- Indeed! We can create a prototype and run usability test. By using a tool like Quant UX, we can have all the information for further improvement of the design.
"On the subject of gamification, did you have any specific thoughts?" -- Yeah!
1)For a volunteer: We can keep a count of the number of published feedbacks. The top 3 volunteers with maximum count will get the reward, let's say, on a monthly basis.
2)For a user(say sharon): Since Sharon is also, in a way, contributing to the community by pointing out issues in the product; gamification can be used to reward her too.
If a volunteer, while writing a feedback, finds that the sharon actually pointed out an "issue" or a "bug" in the app, he/she can add points(say, a star) to that review. After the 3 moderations are successful, sharon will receive that star. And the rewarding process will be same as that of a volunteer.
This way, we can prevent unnecessary feedbacks and reviews.
Thank-you
Thanks for the response @sunidhi-kashyap
One remark regarding:
After the 3 moderations are successful, sharon will receive that star. And the rewarding process will be same as that of a volunteer.
This way, we can prevent unnecessary feedbacks and reviews.
I wonder, whether this can be „gamed with”. Like, I tell some friends that we are going to push each other up to earn the rewards.
Could be, that this will never be attempted, but I feel, it's something which should be at least took into consideration.
@Ryuno-Ki Yeah I totally get your point. We may need to do a little research on this, I guess, before coming into conclusions.
Thankyou for pointing it out.
I know I am very late, but since the issue is not yet closed I made a paper-pen flowchart of the application.
Can you please review this?
@mcroud Given the layoffs earlier this year: is this tool still on the table?
(If you are allowed to say, that is).
Hi @RiyaJ2311, thank you for producing this experience flow.
I like your use of the term "Discover" it feels quite welcoming to new volunteers, I could imagine each app in the app selection screen having a bit of descriptive text or tips in this scenario. It also breaks out of the overuse of the term "Respond".
The idea of an in-app help form for volunteers dealing with a tough online review is also a welcome suggestion. Perhaps more experienced responders will receive a notification in their inbox if someone has completed the help form and requested assistance.
Thanks for sharing your ideas!
@Ryuno-Ki The Respond tool remains an active project. The new support team is in the process of re-evaluating the Play store support strategy.
I think I can close this project now. A big thank you to everyone that submitted ideas, suggestions and thoughts for this project. As mentioned in my April post, the shared effort in this thread has been enthused in a few organisational meetings, thank you for making it such a success!