Timeline
~12 months for full MVP
My Role
Researcher, Design & Developement
Deliverables
iOS App, Research paper, Qunatitaive Data Collection & Qualitative Analysis
Project Summary
This research evaluated different modalities for turning pages in digital sheet music during piano performance. As more musicians adopt tablets over traditional printed music, the challenge of page turning while playing has become increasingly disruptive. The study compared touch-based (swipe, tap) and hands-free (foot pedal, head gesture, wink) page turning methods through a controlled lab experiment, measuring usability, user experience, and performance. The findings showed hands-free modalities, especially head gestures, were generally preferred over touch-based options, providing insights to inform the design of more intuitive and user-friendly digital sheet music interfaces.
What are the problems?
Musicians, especially pianists, face significant challenges when turning pages of digital sheet music while performing, as their hands are occupied playing the instrument. Traditional page turning methods, such as touching the screen or using a foot pedal, can disrupt the flow of performance and add cognitive load. There is a need to explore alternative, more seamless page turning modalities that better support the unique needs and constraints of musicians when working with digital musical scores.
Approach & Design Process
I conducted a controlled laboratory study to evaluate different page-turning modalities for digital sheet music during piano play. The study involved 15 participants, including 12 intermediate and 3 professional pianists, and tested five methods: swipe, tap, foot pedal, head gesture, and wink. All participants played the standardized musical piece, La Valse d'Amélie, under each experimental condition. To gather data, the researchers measured the duration of each page-turning action and collected user experience ratings through the standardized User Experience Questionnaire (UEQ). Additionally, semi-structured interviews were conducted to gather qualitative insights from participants. The data analysis included statistical evaluation of the quantitative results to identify significant differences between the modalities, while affinity diagramming of the interview transcripts helped extract key themes. By synthesizing both quantitative and qualitative findings, we concluded about the usability and user experience of the various page-turning methods. This mixed-methods approach provided a comprehensive evaluation of the modalities' suitability for musicians working with digital sheet music.
Highlights
1. Hands-free modalities (head gesture, foot pedal, wink) were generally preferred over touch-based methods (swipe, tap). 2. Head gestures emerged as a promising alternative to foot pedals, particularly in terms of perceived attractiveness, efficiency, stimulation, and novelty. 3. Touch-based methods were rated highest for perspicuity but lowest for efficiency and novelty Wink gestures were perceived as interesting but less dependable due to the challenge of controlled eye movement. Contextual factors such as performance setting, familiarity with the piece, and individual preferences influence the choice of page turning method
Key Metrics
Design Guide
From participants' comments during the interviews, we were able to obtain further insights as to how they perceived the different page turning modalities. Regarding personal preference, 10 of the fifteen participants preferred the head modality over other page turning modalities, while three participants preferred the foot pedal. The tap and wink modalities were preferred by one participant each. Touch-based modalities such as tap and swipe were generally considered straightforward and reliable, but all sixteen participants highlighted the major disadvantage being that these modalities required moving the hand away from the keyboard. Three participants, who were intermediate players, highlighted that they were sometimes having a hard time repositioning their hands on the keyboard after touching the screen. P2, being a professional player, confirmed that it is typical for beginners to look at the keyboard while playing, which might cause a loss of context and orientation when switching visual focus back and forth between the piano keyboard and the sheet music. Four participants highlighted that it depends on diverse factors whether pianists can put their hands off the keyboard, such as the general difficulty of a piece, familiarity with a piece, or how the music notes are laid out (especially at the end of a page). P5 highlighted that for sight-readers and experienced pianists, it is common to read four bars in advance, which allows for taking the time to put one's hands off the piano to turn the pages. In direct comparison, six participants preferred tapping over swiping because it is a shorter and less complex movement. In contrast, three other participants mentioned that they found tapping harder than swiping because they needed to target a specific screen area, whereas swiping was possible anywhere. Three other participants mentioned that they found swiping more natural because, for musicians who are used to turning pages physically, swiping is more or less the same movement -- only just on a smaller scale.
Conclusion
In this paper, we evaluated five modalities for turning the pages in digital sheet music during piano play. Our results show that the participants favor the head modality due to the perceived efficiency and sense of control. Besides that, the foot pedal was highlighted as the most feasible hands-free page turning modality, but with the limitation of potential conflicts with the sustain pedal. The wink modality was identified as less promising due to the perceived additional burden of controlled eye movement. We hope that the present work has implications for future designs and investigations of gesture-based input modalities in music performance.
For future work, we plan to further investigate and improve hands-free modalities based on head tracking or facial gestures.
Furthermore, we are interested in exploring multi-modal page turning modalities to improve tracking robustness, or provide customizable choices based on context and individual preference.
While the present study focused on the field of piano play, we are also interested in replicating the concept for other musical instruments.
