Testing our Prototype and the Observation Quantitative Analysis

As a prototype is meant to be a reduced version of our final product, it is necessary to get a number of people to test the prototype and record the results of their experience.

As I was travelling for the final two weeks of the assignment, I asked three of my colleagues in our Colorado office to test the prototype. This emphasized the importance of using an online prototyping tool, as it allowed me to carry out the interviews without needing the paper, as Jacinta and I were in two different locations.

Here’s the tasks we are asking users to perform when testing the app. These are directly based on our job stories. It was important to first give the interviewees the context of why they were performing this task, and what the application was.

  • I want you to find a coffee shop which currently has deals on
  • I want you to find the specific deal which is on in that coffee shop
  • I want you to rate the coffee shop.
  • I want you to go the home screen
  • I want you to update your coffee preferences, to include alternative Milk options.
  • I want you to see all coffee shops in the area.
  • Find a coffee shop which accepts credit cards.

Interview 1 (E. Chouinard):

Tasks

  • I want you to find a coffee shop which currently has deals on

Observation Notes: Clicked on free word rather than on icon, but got it quickly.

  • I want you to find the specific deal which is on in that coffee shop

Observation Notes: Looked for text first but then notice “free” star pretty quickly

  • I want you to rate the coffee shop.

Observation Notes: Tried to click the stars, rather than the heart. Really struggled here.

  • I want you to go the home screen

Observation Notes: This was fine, clicked done in right spot.

  • I want you to update your coffee preferences, to include alternative Milk options.

Observation Notes: Went smoothly

  • I want you to see all coffee shops in the area.

Observation Notes: Didn’t really understand logo, but through trial and error got it.

  • Find a coffee shop which accepts credit cards.

Observation Notes: This took a couple of go’s but found it eventually. Clicked one of the purple icons also.

Interview 2 (S Nelson):

Tasks

  • I want you to find a coffee shop which currently has deals on

Observation Notes: Noticed the free, and clicked on that first

  • I want you to find the specific deal which is on in that coffee shop

Observation Notes: Got this after thinking for a few seconds. Wasn’t sure about why they should have to click “Free”, should it not display offer

  • I want you to rate the coffee shop.

Observation Notes: Tried to click the stars, rather than the heart. Then clicked star and clicked done.

  • I want you to go the home screen

Observation Notes: All good

  • I want you to update your coffee preferences, to include alternative Milk options.

Observation Notes: All good

  • I want you to see all coffee shops in the area.

Observation Notes: Got this, but didn’t understand purple versus black?

  • Find a coffee shop which accepts credit cards.

Observation Notes: Got completely stuck on this one. I had to help. Clicked on bond street, and didn’t see symbol (but didn’t know what symbol it would be). Then clicked on George Street but Bond Street was displayed.

Other feedback

Why would we need to clicked on all of them to get credit card , it should only display cards accepted… Based on preferences.

Felt like she was a good test subject as ye wouldn’t normally use apps

Interview 3 (M. Jach):

Mike sailed through 1,2,4,5,6 in seconds.

3: I want you to rate the coffee shop.

Notes: Tried to click stars on rating also.

7: Find a coffee shop which accepts credit cards.

Notes: But got completely stuck on 7, similar to Stacy on problems. I had to help after watching Michael click around for a few minutes (clicked on search icon a lot)

Summary of Observations to meet Quantitative Goals

Lets remind ourselves of our quantitative goals before we summarize observations.

  • Users can find coffee deals in less < 3 seconds.
  • Users can update their coffee preferences in <  6 seconds.
  • Users can rate a coffee shop in less the < 5 seconds.

The observations were mainly positive, with goals 1 and 2 above achieved in 66% of the three test cases, and goal 3 achieved in 33% of the test cases.

Tasks 1,2,4,5 and 6 were largely done quickly, but 3 and 7 causes issues.

3: I want you to rate the coffee shop.

The presence of blank stars on the rating screen caused mass confusion, with lots of clicks. This is seen as an easy fix in our second iteration of the prototype.

7: Find a coffee shop which accepts credit cards.

Although this wasn’t one of our quantitative goals, we expect it will impact our qualitative results in the next blog post. The users struggled to find the coffee shops that accepted credit cards, as they didn’t know the symbol, or even why that coffee shops that dont are displayed in my preferences.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s