Protected: Reflection

This content is password protected. To view it please enter your password below:

Advertisements

Adjusting the Prototype based on the observations and feedback

An important step in the UX process is to implement the improvements from your learnings, based on your own observations, along with user feedback. Whilst making widespread changes to the prototype was outside the scope of this project, it was necessary to at least go through this process. The second iteration of the prototype was at the below link.

https://share.proto.io/B79LYF/

The two visual changes are shown below (the actual physical features are not, due to time constraints), with us having a more visual on/off on preferences so that the user can see which shows their preferences.

We also put the offer into the star, as people that used the prototyped found that this slowed them down in finding the offer (why hide it in an image).

1-filter-popover3-filter-profile

Interpreting the Prototype Session Feedback

For our qualitative goals, we used a survey as already discussed to get a sense of the user experience. The goals are below again for reference, with analysis in groupings based on the two goals..

1 = strongly disagree 2 = strongly agree

Qualitative

  • Users find the app easy to use.
  • Users find the app easy to learn.
1 2 6

Qualitative Goal 1 Analysis: Large disagreement with these three questions, so it is a good overall question to gauge the summary of results for  our first qualitative goal. Goal met.3457

Qualitative Goal 2 Analysis: Again, strong positive results here for our second qualitative goal. The second result is interesting, as it is more spread out in terms of result when compared to the other questions. I expect this as the movement between screens is not perfect is design, but might appear clunky to go from one task to the next (as opposed to the individual tasks) Goal met.

Largely, this should be seen as a positive result, but caution should be aired. The sample set is quite small (8 people), so it would be interesting to do a larger number of interviews after the second version of the prototype.

Capturing the Prototype Testing Feedback

Once the subjects had completed the testing of the prototype using proto.io, we asked the subjects to fill in a very short survey more to gauge the success that we had at meeting our qualitative goals. The survey is at the below link, along with the questions. We asked these seven questions to ensure we had enough angles at the two simple qualitative goals we asked, which were was the app easy to use, and easy to learn.

https://docs.google.com/forms/d/e/1FAIpQLScGApf834PFnMnIvUcnvxjZO_OqAnxodcJxrWKjEqWWSPTjbg/viewform?c=0&w=1

I found the product unnecessarily complex.
Strongly disagree                                  Strongly agree
1                   2                       3                   4                        5
 
I thought the product was easy to use
Strongly disagree                                  Strongly agree
1                   2                       3                   4                        5
I think that I would need the support of a technical person to be able to use this product.
Strongly disagree                                  Strongly agree
1                   2                       3                   4                        5
I  found the various functions in the product were well integrated
Strongly disagree                                  Strongly agree
1                   2                       3                   4                        5
I imagine that most people would learn to use this product very quickly.
Strongly disagree                                  Strongly agree
1                   2                       3                   4                        5
I found the product very awkward to use.
Strongly disagree                                  Strongly agree
1                   2                       3                   4                        5
I needed to learn a lot of things before I could get going with this product.
Strongly disagree                                  Strongly agree
1                   2                       3                   4                        5

Testing our Prototype and the Observation Quantitative Analysis

As a prototype is meant to be a reduced version of our final product, it is necessary to get a number of people to test the prototype and record the results of their experience.

As I was travelling for the final two weeks of the assignment, I asked three of my colleagues in our Colorado office to test the prototype. This emphasized the importance of using an online prototyping tool, as it allowed me to carry out the interviews without needing the paper, as Jacinta and I were in two different locations.

Here’s the tasks we are asking users to perform when testing the app. These are directly based on our job stories. It was important to first give the interviewees the context of why they were performing this task, and what the application was.

  • I want you to find a coffee shop which currently has deals on
  • I want you to find the specific deal which is on in that coffee shop
  • I want you to rate the coffee shop.
  • I want you to go the home screen
  • I want you to update your coffee preferences, to include alternative Milk options.
  • I want you to see all coffee shops in the area.
  • Find a coffee shop which accepts credit cards.

Interview 1 (E. Chouinard):

Tasks

  • I want you to find a coffee shop which currently has deals on

Observation Notes: Clicked on free word rather than on icon, but got it quickly.

  • I want you to find the specific deal which is on in that coffee shop

Observation Notes: Looked for text first but then notice “free” star pretty quickly

  • I want you to rate the coffee shop.

Observation Notes: Tried to click the stars, rather than the heart. Really struggled here.

  • I want you to go the home screen

Observation Notes: This was fine, clicked done in right spot.

  • I want you to update your coffee preferences, to include alternative Milk options.

Observation Notes: Went smoothly

  • I want you to see all coffee shops in the area.

Observation Notes: Didn’t really understand logo, but through trial and error got it.

  • Find a coffee shop which accepts credit cards.

Observation Notes: This took a couple of go’s but found it eventually. Clicked one of the purple icons also.

Interview 2 (S Nelson):

Tasks

  • I want you to find a coffee shop which currently has deals on

Observation Notes: Noticed the free, and clicked on that first

  • I want you to find the specific deal which is on in that coffee shop

Observation Notes: Got this after thinking for a few seconds. Wasn’t sure about why they should have to click “Free”, should it not display offer

  • I want you to rate the coffee shop.

Observation Notes: Tried to click the stars, rather than the heart. Then clicked star and clicked done.

  • I want you to go the home screen

Observation Notes: All good

  • I want you to update your coffee preferences, to include alternative Milk options.

Observation Notes: All good

  • I want you to see all coffee shops in the area.

Observation Notes: Got this, but didn’t understand purple versus black?

  • Find a coffee shop which accepts credit cards.

Observation Notes: Got completely stuck on this one. I had to help. Clicked on bond street, and didn’t see symbol (but didn’t know what symbol it would be). Then clicked on George Street but Bond Street was displayed.

Other feedback

Why would we need to clicked on all of them to get credit card , it should only display cards accepted… Based on preferences.

Felt like she was a good test subject as ye wouldn’t normally use apps

Interview 3 (M. Jach):

Mike sailed through 1,2,4,5,6 in seconds.

3: I want you to rate the coffee shop.

Notes: Tried to click stars on rating also.

7: Find a coffee shop which accepts credit cards.

Notes: But got completely stuck on 7, similar to Stacy on problems. I had to help after watching Michael click around for a few minutes (clicked on search icon a lot)

Summary of Observations to meet Quantitative Goals

Lets remind ourselves of our quantitative goals before we summarize observations.

  • Users can find coffee deals in less < 3 seconds.
  • Users can update their coffee preferences in <  6 seconds.
  • Users can rate a coffee shop in less the < 5 seconds.

The observations were mainly positive, with goals 1 and 2 above achieved in 66% of the three test cases, and goal 3 achieved in 33% of the test cases.

Tasks 1,2,4,5 and 6 were largely done quickly, but 3 and 7 causes issues.

3: I want you to rate the coffee shop.

The presence of blank stars on the rating screen caused mass confusion, with lots of clicks. This is seen as an easy fix in our second iteration of the prototype.

7: Find a coffee shop which accepts credit cards.

Although this wasn’t one of our quantitative goals, we expect it will impact our qualitative results in the next blog post. The users struggled to find the coffee shops that accepted credit cards, as they didn’t know the symbol, or even why that coffee shops that dont are displayed in my preferences.

Our Goals

Before we test our prototype its important to see a series of goals to test against. Goals can be other quantitative or qualitative. The goals we chose are below.

Quantitative

  • Users can find coffee deals in less < 3 seconds.
  • Users can update their coffee preferences in <  6 seconds.
  • Users can rate a coffee shop in less the < 5 seconds.

Qualitative

  • Users find the app easy to use.
  • Users find the app easy to learn.

Our Prototype

We choose the below job stories to be in our prototype as there were a number of items which resonated with the people in our surveys and in our the interviews.

  • When I am away, I want to know where does good coffee, so I can drink coffee I like
  • When I am in a good cafe, I want to be able to review the experience, so I can let my friends know if its good or not
  • When I want a new coffee experience, I want to know what alternative milk types are present, so I can ensure my lactose intolerance needs are met
  • When I want a coffee but have no cash, I want to know which coffee establishments have alternative payment options, so I can prepay or pay with card.
  • When I want to save money, I want to know which of my preferred coffee shops are giving offers, so I can stay within my household budget.

For our prototype, we started with some paper prototypes to begin with. We felt it would be good way to start, and also allow us to brainstorm about the look and feel of how the prototype. Some images of this are below.

Pretty quickly we began to see how it might be difficult to do our gorilla user testing with paper prototypes are they are not interactive, and the user would need more direction, which can be a distraction.

We decided to go digital, and use proto.io to build our prototype. We believed it would allow the user a more fuller experience so that they could fully test and walk through the application at a reasonable level of interactivity. The link to this first iteration of our prototype is below, along with the associated screen captures.

https://pr.to/44MJHJ/

1-screen-12-filter-popover3-starbucks-popver4-starb-profile5-filter-rate-me6-screen-17-filter-profile8-user-profile

A Job Storyboard

To complete the picture, I decided to storyboard one of our job storyboards as an exercise before we decide on our prototype solution. The job story I prototyped is below. For this storyboard, I decided to storyboard the solution in action, rather than before having the solution, which was done for the user storyboard.

When I want a coffee but have no cash, I want to know which coffee establishments have alternative payment options, so I can prepay or pay with card.

Storyboard4.png

As I went through this storyboard, it is easy to see the “job to be done” by the solution, and it reminded me of Clay Christensen’s YouTube video on the milkshake job to the done (I just completed the Innovator Accelerator program). What do we want the app to do or solve? This would need to be one of the job stories for our prototype build.

Our Job Stories

From completeness, we also converted our user stories into its peer format, which are job stories. We felt it would be a useful exercise to increase our understanding of the difference between them, as it is expected that the product you try to design might be better suited to one of these rather than the other.

When _____ , I want to _____ , so I can _____ .

Job Stories

  • When I meet friends, I want to get coffee recommendations, so I can meet my friends in a coffee shop we both like.
  • When I am away, I want to know where does good coffee, so I can drink coffee I like
  • When I am in a good cafe, I want to be able to review the experience, so I can let my friends know if its good or not
  • When I am away from home, I want to be able to get coffee recommendations from like minded people, so I can ensure I have an enjoyable experience
  • When I am with my children, I want to know which coffee shops have child friendly facilities, so I can ensure my kids needs are met
  • When I want a new coffee experience, I want to know what alternative milk types are present, so I can ensure my lactose intolerance needs are met
  • When I want a coffee but have no cash, I want to know which coffee establishments have alternative payment options, so I can prepay or pay with card.
  • When I want to save money, I want to know which of my preferred coffee shops are giving offers, so I can stay within my household budget.
  • When I am free to have coffee, I want to know which of my friends are about to go for coffee, so I can catch up with them.
  • When I am new to a place, I want to know which coffee shops are sociable, so I can meet people of similar interests

I can see how we could use both user stories and job stories together. User stories are better for defining features of a solution, whereas job stories are better for defining high level goals.

By defining the job stories first (with goals), we can then create a prioritize multiple solutions with user stories to solve these goals.

A User Storyboard

Once the user stories were defined, I decided to storyboard one of the user stories. A storyboard uses a series of sketches in a comic book fashion to describe the story that a user might go on in the story.

We used http://www.storyboardthat.com/ to create the storyboards. The user story I decided to storyboard is below.

“As a busy person, I want to know which of my friends are about to go for coffee, so that I can combine two things I love – catching up with friends and great coffee”

firststoryboard

As I went through this process, I can see how it is important to keep your mind focused on your persona Peter rather than your own vision of the story. It also introduced subtleties around the reasons why Peter would want to meet colleagues for coffee, to share recommendations,  get information on projects for example.