Introduction

This report looks at the playlist creation workflow within the apple music app (IOS), focusing on 3 sub tasks: creating a new playlist, adding 5 songs and reorder the tracklist. Playlist creation is a core feature and everyday interaction within the app, which makes it a useful benchmark on evaluating the app over usability (Hassenzahl, 2010). Six participants were all tested seperatly and told the task before the timer was set. All sessions were screen recored with audio transcription and monitored live, to capture there interactions with Apple Musics UI Design. This gave enough data to breakdown where the interface supports the user and where it might break down.

Method Data Collection

All participants were asked to 'think aloud' during the in person testing, narrating there actions and thought process behind each action. This is so any moment of confusion or discovery was easily identifiable. Each participant was told an identical task script before beginning: Create a playlist called "Road Trip", Add 5 songs to the playlist, and reorder the playlist so a favourite track is first. With no other instructions given to the participants, it leaves the apps interface to guide the user (NNgroup, 2018).

Each session was screen recorded with live microphone audio captured, syncing the verbal commentary in time with the users actions. The screen recording were then reviewed frame by frame in order to build a timeline and timestamps of each sub-task duration. By collecting these two points of data for each participant (visual and verbal), users behaviour and reasoning could be reviewed in tandem.

Participant Profile

Findings - data from each participant

Six Participants were recorded for this analysis, all varying in age, music app experience, and chosen mobile operating system. This created quite a wide spectrum of completion times, see table and graph below. Participant 2 (p2), P3, P4 and P6 all completed the full task under 104 seconds, all broadly following the task of creating the playlist, adding songs and reordering. The remaining two other participants Participate 1 and Participate 5, took marginally longer (360 sec and 287 sec respectively) for reasons mentioned below.

Participant 1 (Alssya)

P1 spend a disproportion amount of time on the song adding sub task. After talking to the participant after the analysis, they stated that they were distracted by what songs to add to the playlist. P1 ending up spending 42% of there total time on the song add task alone.

Participant 5 (Micheal)

P5 is were the user interface fails the hardest. Instead of adding 5 songs to the playlist, the user added a another playlist (titled "brown sugar") to their playlist. Although this is technically correct, the app failed to give him any confirmation on what they had just done. It wasn't until i had to intervene myself to let the user know they had added 5 songs. Recognising something had gone wrong the user was left question what they had done wrong, quote "Why? Did i just add a record or something". Moving on to the next sub task (playlist reordering), is where the user struggled the most, unable to reliably find the drag handle mechanism within the user interface. Image: Notification showing song have been added

Participant 6 (Lucibella)

P6 had the cleanest run out off all the participants, while they was not the fastest, I saw the least amount of misclicked from them. They named the playlist correctly, added all 5 songs and located the reorder handle immediately with zero confusion verbalised.

Participant 2 (Stephanie)

P2 was the fastest out of all the contestants to create the playlist and add 5 songs. Reordering was their only point of friction, trying to long press and drag the song without opening edit mode. They then attempted to sort again but instead just ordered it in alphabetical order. It wasn't until the 3rd attempt they were able to find the edit menu and correctly reorder the playlist. This last sub-task took 66 seconds - 71% of her total time. Image: Highlighting drag handles to reorder songs

Participant 4 (Fred)

P4 had a relatively accurate run, with only one accidentally tap, starting a shuffle play when trying to reorder songs. They recovered from this swiftly without any intervention, finding the context menu to edit the playlist.

Participant 3 (Kean)

This participant completed the task the fastest out of everyone overall, following the create playlist, add songs from search and edit. This user still struggled to find the edit mode for the playlist though, just like the rest of the participants.

Graphed Data

Analyse - patterns across participants

Two patterns stand out when comparing all 6 participants side by side

The first being the results being bimodel rather than a gradual shift . Four participants fall between the 73 - 104 second time range, while the other 2 sit between 287s and 360s. This suggest that user will either fall into two categories, people who can use the app with minimal friction or those who hit a spot that derails the entire task. This distinction shows how the app can be improved by fixing small points of failure points rather than a general overhaul of the interface.

The second pattern is that reorder is a struggle no matter the user. P2 completed task 1 and 2 with ease (under 30 sec, the fastest out of anyone), but then spent 66 seconds on trying to reorder the playlist (71% of her task time). This combined with the other participants results is evidence that the reordering interface is flawed from a usability perspective.

The two outlier participants further reinforce this criticism, but each separately demonstrate a design failure. P1 slowdown stemmed from clicking add song, this put the user on a familiar screen that was similar to app main menu. Left on this page, P1 wasn't sure what happened and if they had clicked the right thing. This highlights the lack of context given to the user when put into this menu, the interface needs a stronger signal so the user can better understand what is happening.

P5 is arguably the most serve usability issue found within the whole test. This user wanted to add one song from a playlist and clicked the arguable ambiguous "+" button that would usually just add one song to his playlist. But because he was within a sub-menu of another playlist, they ended up adding a whole seperate curated playlist to their own playlist. The app also failed to give confirmation dialog to the user, making the action known.

P6 clean results is an important counterpoint to this argument, demonstrating that the reorder interaction is not hard to discover but hard to use. Even though this user regularly uses this app on a daily basis, they too had trouble with the gesture required to reorder the playlist.

Participants profiles data partially explains the split within this data. P3 and P4 are both android user but had the fastest times out of anyone, while P1 a native iOS user produced the slowest time. Preferred music streaming app produced a cleaner result with P6 but this doesn't apply to the other non-apple music, who were able to achieve a similar result. Age is a more notable factor within this data, P5 being 51, making them 15 years older than the next participant, and almost 20 years above the average. While this single data point cannot prove anything specific, it is important to pay attention to. P5 session was plagued with difficultly throughout the entire app, rather than one or two failure points. This raises the age related argument rather then the app I specifically being unfamiliar.

With all the evidence considered, two critical design flaws have been identified. The first being the drag to reorder gesture is hard to discover, with low visual context and the gesture itself being hard to use. The second being the "+" button behaviour being reliant on the menu context, meaning that in one place in the app it might add a song to a playlist or in another section it might download a group of songs. If the user was given more context after clicking this button, the user might be able to learn from their action but that doesn't exist within the app.

Reflection - How different data types informed analysis

Timing data while useful was not enough to indicate how and why something when wrong. P5 task times look faily even from a glance, failing to indicate what went wrong. It's not until the time data is cross referenced with the audio transcript and screen recording that we able to identify the error made within the interface. Video helped to confirm exactly what happened on the screen and audio captured the participants confusion and reasoning in real time. Timing data just allowed the anomaly to be visible within the data, by standing out when compared to the rest of the participants.

The test methodology did have limitation that are worth nothing such as,

• Hints were given based off the participants difficulty to complete the task
• iPhone used to test participants would slow down/lag at times unpredictability

References

Hassenzahl, M. (2010). User experience and experience design. UX Courses. The Interaction Design Foundation. https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/user-experience-and-experience-design

NNgroup. (2018, November 9). How to test visual design [Video recording]. In YouTube. https://www.youtube.com/watch?v=FcguM7pFjI8