Coloring-in Piano: Indiscrete Musical Elements are Essential for Performers
Chika Oshima+, Kazushi Nishimoto+*, Yohei Miyagawa+, and Takashi Shirosaki+
+School of Knowledge Science, Japan Advanced Institute of Science and Technology *PRESTO, JST email: cooshi@jaist.ac.jp

Abstract

In this description, we propose a new musical instrument named the CiP (Coloring-in Piano) that can facilitate to externalize musical expression of performers. Conventional music consists of two kinds of elements, i.e., "discrete elements" and "indiscrete elements." Pitch of each note and basic rhythm of the notes (i.e., time value of each note) described in a score are the discrete elements, while tempo, "Artikulation," "Agogik," "Dynamik," and so on, are the indiscrete elements. CiP releases the performers from a non-creative task to operate one of the discrete elements, i.e., the pitch. As a result, CiP allows the performers to concentrate on creative task, that is, control of the indiscrete elements. Moreover, CiP allows the performer's simultaneous operation of all of the indiscrete elements. We provide three performance data to demonstrate effectiveness of CiP.

Introduction

We propose "Coloring-in Piano (CiP)" that can facilitate to externalize musical expression of performers. CiP releases the performers from a great labor to arrange pitches to reproduce a melody and from making miss-touches, and allows them to immediately concentrate on musical expression that is essential for the performers of music. Conventional music consists of two kinds of elements, i.e., "discrete elements" and "indiscrete elements." Pitch of each note and basic rhythm of the notes (i.e., time value of each note) described in a score are the discrete elements, while tempo, "Artikulation," "Agogik," "Dynamik," and so on are the indiscrete elements (Nishimoto, et al. 2002). Usually, the performer of the piano is apt to concentrate on reproduction of sequence of notes described in the score. There are many systems that support a novice performer to operate correct keys for accurate reproduction of melodies (YAMAHA 2000)(Casio 2001)(YAMAHA 2001). However, what is necessary and essential for the performer is to express her/his individual interpretation and impressions of an opus than the accurate reproduction of the sequence of notes.
The expression of the performer can be represented by performance on indiscrete elements, not by operations of the discrete elements. Although the composers had come to describe the expression marks minutely in the scores for the performers since about 1830 (Kuniyasu 1981), G. Gould and N. Goodman pointed out that what is eventually described in the score are only the pitch and basic rhythm of each note (Bazzana 1997). Consequently, the musical expression should be created by the performers. Indeed, the expression marks and so on are useful clues to demonstrate musical expression. However, they are not sufficient. Only by the clues provided in the score, it is impossible to demonstrate good musical expression. At least, musical knowledge based on musical experiences, education and studies are required. On the other hand, heavy labor to accurately reproduce the given melodies is imposed on the performer. This task prevents the performer from freely externalizing his/her musical expression. Accordingly, we propose a new musical instrument CiP that releases the performers from a noncreative task to input one of the discrete elements, i.e., the pitch. As a result, CiP allows the performers to concentrate on creative task, that is, control of the indiscrete elements. Moreover, CiP allows the performer's simultaneous operation of all of the indiscrete elements. For example, even if only the "crescendo" is described on the score, the performer usually controls not only the volume of sound but also all of the indiscrete elements, e.g., Artikulation, Agogik, tempo. All of the indiscrete elements are tightly related, and the balance among them is very important for achieving good musical expression. Therefore, it is not a good way to divide the set of indiscrete elements into several parts like the step-bystep input method of MIDI (Musical Instrument Digital Interface) data that is employed most of the MIDI sequencing systems. Allowing integrated control of them is important.
Set up of Coloring-in Piano
Figure 1 shows the set up of CiP. CiP consists of a MIDI keyboard, a music-database, a function for replacing note numbers, and a tone-generator. Before
performing, it is necessary to enter the sequence of MIDI note-numbers (corresponding to pitches) of the piece to be performed into the music-database. While performing, the replacing function replaces the played note-numbers with the note-numbers registered in the music-database, based on the order in which they were entered. Accordingly, the correct note number is always output by touching any key. On the other hand, the expressive elements, i.e., noteon (key down) velocity, note-off (key up) velocity, onset/offset timing, and pedal messages, are output as the performer plays. Consequently, the replaced pitch numbers are input into the sound generator with the expressive elements preserved as they were performed. We used a YAMAHA silent grand piano C5 professional model that outputs MIDI (Musical Instrument Digital Interface) note-on/off and pedal control messages. The piano is connected to an SGI Indy workstation in which the music-database and the replacing function of note-numbers are implemented.

composed by F. Chopin, by two methods, i.e., the two-step input method and the real-time input method. She had never played this piece on the piano. In the real-time input method, the sequence data are constructed by performing a normal piano (but that can output MIDI data) in a usual manner and by recording the output MIDI data by using sequencing software. In the two-step input method, she was allowed to use any fingers and any keys as she liked. At first, we gave her five minutes to interpret this piece without a musical instrument. Then, she recorded the MIDI sequence data of the piece by the real-time input method without practice. After that, she practiced the piece in ten minutes on the conventional piano. Then, she recorded the MIDI sequence data of the piece again by the real-time input method and by the two-step input method successively. All of them are performed in almost the same tempo.
5 The difference among three MIDI sequence data
By listening to the three sequence data, you can find that the first performance recorded by the realtime input method includes many mistakes. Moreover, since C. Oshima has never performed this piece, she could not immediately perform it with reasonable expression. The second performance input by the real-time input method is still a little awkward though she practiced the piece for ten minutes. On the other hand, the performance obtained by the two-step input method is more expressive and smooth than the performances by the real-time input method.
Figure 1. Structure of CiP

6 Discussions

These results demonstrate that the two-step input method is easier than the real-time method. Moreover, it is also easier for the people to represent her/his desired expression by the two-step input method than by the real-time input method. We think these advantages of the two-step input method derive from two concepts, i.e., a division of musical elements into two elements and an integrated operation of the indiscrete elements. By dividing the musical elements into two kinds of the elements, i.e., the discrete elements and the indiscrete elements, and by the computers executing the discrete elements, the people can construct the MIDI sequence data easily, no matter how technical the piece is. Furthermore, the precedence of inputting discrete elements can help the people concentrate on indiscrete elements that are essential for musical expressive performance. In another aspect, the integrated operation of the indiscrete elements allows the people to keep a balance among the indiscrete elements. We already investigated the effectiveness of the two-step input method by experiments with 18 subjects (Miyagawa, et al. 2001) (Nishimoto, et al.

3 Two-step input method

We propose a new method named two-step input method for constructing MIDI sequence data. In this method, the process of constructing MIDI sequence data is divided into two steps. In the first step, only the sequence of pitches is input without giving consideration to the velocity and the onset/offset time data: this can be done in a similar manner to the stepby-step input method. Then, in the second step, the expression of the piece, i.e., the velocity and onset/offset time, are input with using CiP by using the sequence of pitches input in the first step. Finally, the performance obtained by integrating the sequence of the pitches and the expressions input by CiP are recorded by sequencing software in a similar manner to the real-time input method.
4 Constructing MIDI sequence data by two input methods
We created three sequence data. C. Oshima, who is one of the authors, tried to construct the MIDI sequence data of a part of "Sonate No.3," which is
2002). The subjects were asked to construct MIDI sequence data by three input methods, i.e., the realtime input method, a step-by-step input method, and the two-step input method. The results suggest that the two-step input method is suitable for the subjects inexperienced in playing a keyboard. Furthermore, most of the subjects were more satisfied with the MIDI sequence data constructed with the two-step input method than that constructed with the real-time input method. Then, by alleviating the cognitive load in performance imposed by non-expressive elements, i.e., the sequence of pitches, we can say that it becomes easier for people to externalize more of their own musical expression.
These advantages derive from the two concepts, i.e, the division of musical elements into two kinds of the elements, say the indiscrete elements and the discrete elements, and the integrated operation of indiscrete elements.

References

Bazzana. K. 1997. GLENN GOULD: The performer in the work. Oxford: Clarendon Press. (In Japanese) Boulanger. R., and Mathews. M. 1997. The 1997 Mathews Radio-Baton and Improvisation Modes. Proceedings of the International Computer Music Conference. International Computer Music Association, pp.395-398. DOREMI master, YAMAHA, 2001. http://www.yamaha.co.jp/product/pk/products/ezj22/ezj 22.html. HIKARI NAVIGATION, Casio, 2001. http://www.casio.co.jp/emi/key_lighting/ Kuniyasu. H. 1981. A guide to a musical aesthetics. Japan: Shunjusya Press. (In Japanese: This book does not have the English title. Therefore, we translated the title from Japanese.) Miyagawa. Y., Shirosaki. T., Oshima. C., and Nishimoto. K. 2001. Two-Step Input Method of MIDI Sequence Data by Using Coloring-in Piano. IPSJ SIG Notes, Information Processing Society of Japan, Vol.2001, No.125, pp.21-26. Nishimoto. K., Oshima. C., Miyagawa. Y., and Shirosaki, T. 2002. A Musical Instrument for Facilitating Musical Expressions. CHI2002, Minneapolis, Minnesota, April 20-25. (to appear) Silent ensemble grand piano EA1, YAMAHA, 2000. http://www.piano-gc.co.jp/catalog1.htm. Takeuchi. Y., and Katayose. H. 1995. Representation of Music Expression with Two Finger Piano. IPSJ SIG Notes, Music and Computer, No.011-066.

Related Works

Many studies have attempted to support a performer to play music by a computer. RadioBaton (Boulanger and Mathews 1997) is a controller for live computer music performances. In the conductor mode, the baton plays a score that is loaded into the processor beforehand, according to the conductor-gestures made by the performer. The conductor may be able to control Dynamik and tempo in musical parlance. The Two Finger Piano (Takeuchi and Katayose 1995) is a toy system that allows people to perform without the need to reproduce accurate pitches with handling tempo and Dynamik for each beat (not for each note) by two fingers. However, the expressive musical elements, i.e., Dynamik, tempo, Artikulation, and Agogik must be controlled in each-note-level. The CASIO LK-40 Lighted Keyboard (Casio 2001) is equipped with a 3-Step Teaching System, which is a self-study system of the keyboard. The first step of this system is quite similar to CiP in always outputting only the available pitches by hitting any key, although it always outputs the constant velocity value. On the other hand, CiP not only meets people from the difficulty of accurately reproducing melodies, but also can control all expressive musical elements in each-note-level.

Composed MIDI Files

1. 2. 3. First trial by the real-time input method without practice Second trial by the real-time input method after ten minutes practice First trial by the Coloring-in Piano just after the second trial by the real-time input method

8 Conclusions

In this description, we proposed the Coloring-in Piano (CiP) that allows the people concentrate on operation of indiscrete elements that are essential for musically expressive performance, without heavy burden of needing to accurately reproduce the sequence of pitches. Additionally, we proposed the two-step input method for facilitating construction of MIDI sequence data by using CiP. We prepared three MIDI sequence data of a piano piece. These performances demonstrate that the twostep input method is easier than the real-time input method. Moreover, the two-step input method facilitates the performer to immediately achieve representation of her/his desired expression, which is hard to be achieved by the real-time input method.