Technology Enhanced Learning of Musical Instrument Performance

You practice your music, but no one’s there to help. Now you can have a personal trainer any time.

As any budding pop star knows, learning a musical instrument is difficult. One reason is that teaching is still based on an ancient method.

In the master-apprentice model, students observe and imitate the teacher, who provides feedback. Students must then practise individually, which some find too solitary. Another problem is the long wait for teacher feedback. Advice is disconnected from body movement and audio sensations of the performance, adding to the difficulty. As such, the master-apprentice model leads to high abandonment rates.

The EU-funded TELMI project developed a complementary alternative based on interactive software. Essentially, the software tracks and visually represents the students’ movements, while also monitoring the sound, in both cases giving immediate feedback.

Researchers first surveyed teachers and students of music, representing several prestigious European schools, to determine their requirements. The study also examined the participants’ relationship to technology. The project’s prototype system focused on practice of the violin.

Borrowing from sports

“Musical performance shares many characteristics with other skill-oriented activities,” says Associate Professor Rafael Ramirez, TELMI project coordinator. “For example, commonalities between sports and musical performance are obvious, particularly in the area of biomechanics. So, it’s reasonable to imagine that some methodologies already used successfully in sports kinesiology could be useful in studying aspects of musical performance.”

TELMI’s specially designed sensors are mounted on the bow, tracking its position and speed with respect to the violin. Simultaneously, an off-the-shelf Kinect sensor system tracks whole-body movement. Kinect was developed by Microsoft for the Xbox 360 games system; it tracks movement through a 3D grid, using a combination of infrared/optical projectors and cameras. The TELMI system also uses an electromyogram (muscle sensor) to measure muscular force, and to supplement detection of body position. Additional sensors monitor the music’s audio qualities, which the system graphically represents for the student.

The system instantly compares all sensor data against the project’s purpose-designed database. The publicly available reference contains expert versions of all body position and audio information for numerous musical pieces. The student, either alone or with a teacher, uses the visualised comparison to make necessary corrections. In addition, the system provides feedback on tuning accuracy, timing and musical expression.

An unlikely union

“We are proud of the level of interaction and collaboration we achieved between the technological and educational partners,” adds Prof. Ramirez. Students and teachers involved in the study welcomed the new system into their traditional music teaching environment.

The system is scalable. Users can engage with it in dedicated motion capture rooms in a conservatoire or in their own homes using a laptop. Soon, users will also be able to use the system via mobile phone.

Next, the TELMI team plans to create two spin-off companies to develop and market the project’s technologies. Future systems will be specialised for other instruments too.

The new system removes one of the main weaknesses of the master-apprentice system, without replacing teachers. Now, students can receive informative feedback in real time instead of having to wait for their next lesson. This may improve learning speed and effectiveness, reducing the dropout rate.

Keywords
TELMI, music, musical performance, teaching, biomechanical, master-apprentice model, violin, kinesiology, sensor system

Reference source: Technology Enhanced Learning of Musical Instrument Performance