RHYME member Karette Stensæth has worked very hard for a long time to write and edit the book about RHYME called “Music, Health, Technology and Design”. The book is part of the Book Series from Centre for Music and Health and can be ordered here.
From the back side of the book (see the orange page above):
“Imagine that objects in your home environment – let us say a pillow, a carpet or a toy – became musical and interactive. Do you think that they could offer new ways of playing and being together? Could they even have the potential to reduce isolation and passivity and promote health and well-being for some of us?
This anthology, the eighth in the Series from the Centre for Music and Health, presents a compilation of articles that explore the many intersections of music, health, technology and design. The first and largest part of the book includes articles deriving from the multidisciplinary research project called RHYME (www.rhyme.no). They engage with the study of the design, development, and use of digital and musical ‘co-creative tangibles’ for the potential health benefit of families with a child having physical or mental needs.
Well-known international researchers broaden the picture on the book’s topic in the second part. They ask: How can video-based visualisation techniques of music-related body motion diagnose health problems? How can music therapy practice profit by digitalised improvisation analysis? What are the implications of gender and age in music technology for therapists and the people with whom they work? All together, this book
supplies a broad perspective on its topic, which should be of interest to a wider audience.
The Centre for Music and Health at the Norwegian Academy of Music was established in 2008. The centre conducts research and dissemination. Its goal is to develop knowledge about the connections between music and health.”
Table of Contents
Designing four generations of ‘Musicking Tangibles’
Birgitta Cappelen and Anders-Petter Andersson
Vocal and tangible interaction in RHYME
Anders-Petter Andersson and Birgitta Cappelen
An interactive technology for health:
New possibilities for the field of music and health and for music therapy?
A case study of two children with disabilities playing with ‘ORFI’
Karette Stensæth and Even Ruud
Potentials and challenges in interactive and musical collaborations involving children with disparate disabilities
A comparison study of how Petronella, with Down syndrome, and Dylan, with autism, interact with the musical and interactive tangible ‘WAVE’
‘Come sing, dance and relax with me!’
Exploring interactive ‘health musicking’ between a girl with disabilities and her family playing with ‘REFLECT’ (A case study)
‘FIELD AND AGENT’: Health and characteristic dualities in the co-creative, interactive and musical tangibles in the RHYME project
Health affordances of the RHYME artefacts
PARTICIPATION: A combined perspective on the concept from the fields of informatics and music and health
Karette Stensæth, Harald Holone, and Jo Herstad
From experimental music technology to clinical tool
Alexander Refsum Jensenius
Technology and clinical improvisation – from production and playback to analysis and interpretation
Jaakko Erkkilä, Esa Ala-Ruona, and Olivier Lartillot
Using electronic and digital technologies in music therapy: the implications of gender and age for therapists and the people with whom they work
Wendy L. Magee
Rhyme’s design and development team is currently doing the last tests of the Polly Compose APP before the user studies with the families.
In the Polly Compose APP the user can interact with the Interactive things at home, Polly Planet, Polly Ocean and Polly Fire through a smartphone or a tablet (see photos below). Polly Compose has an easy to use graphical user interface (see the two picture in the middle) and text based (twitter) user interface (see picture right) for more advanced music composition.
Here Anders is testing the tablet interface on an iPad.
First you choose the Polly Compose APP on your desktop. The first, Scenes & Things-screen, of three in the APP opens (see picture to the left).
There you choose Scene (music and visuals) and Thing (Polly Ocean, Polly Fire, Polly Planet). Then hit the [play] button at the bottom (see picture left).
The second screen with the chosen thing in the middle opens (see picture in the middle). Here you can test out the and learn how to play on the thing and how it sounds, before sending the composition to be played on the physical thing at home. You build up a composition by hitting the sensors on the screen representing the sensors on the interactive physical thing. You can also specify how you want the action to be performed (fast, slow, slow etc.). A composition is being built up and visualized over the play and send buttons at the bottom of the screen (see right upper picture).
You play your composition by hitting the [play] button. You send the composition by hitting [send], and you edit (see left lower picture) and save and give it a name by hitting  .
You can go to the first screen and choose another Scene or Thing by hitting [back].
The sent Twitter Composition is received in the Interactive thing. In our example here Birgitta sends a Composition to Polly Planet (see picture lower right) and Polly Planet plays the music and light Composition.
We have created a little video about the physical interaction i Polly World (not the social screen based App interaction). The video is shot by Alexandre Chapell and edited by Mariko Rhode and Birgitta Cappelen. Thanks to Sara, Anders and Berit and their children for their contribution.
RHYME contributed with two keynote lectures at the 10th Nordic Snoezelen Conference 2014 in Helsingør. The two lectures were Birgitta Cappelen’s Designing Next Generation Interactive MSE and Anders-Petter Andersson’s lecture Music in RHYME.
In Birgitta’s lecture she argued based on a democratic and humanistic perspectives and a resource oriented approach for a design of Multi Sensory Environments (MSE) that offers continuous positive experiences and challenging mastery curve for diverse users. This demands a design that is both sensorially stimulating and easy in use and complex at the same time. This demands a computational and software solution that offers many ways to interact and develop actability and mastery over time, not only simple switch solutions.
In Anders-Petter’s lecture Music in RHYME he explained how the musical simplicity and complexity is built up, based on musical sounds and programmed rules that are activated in different user-situations.
The Nordic Snoezelen (also called Multi-sensory Environment - MSE) Conference is a biannual conference and the most important community in the Nordic Countries for use of sensory stimulation in therapy and health promotion.
The Norwegian Community is called Norske Sanserom, the American Community is called AAMSE and the international organisation ISNA. Within MSE very little has been done regarding use of advanced computational possibilities, interactive music and professional design and aesthetic competence when creating the Multi-Sensory Environments. In this context RHYME is a unique project internationally. If you want to read more about MSE research you will find a list here.
On 3rd July RHYME member Anders-Petter Andersson made a presentation of a paper by himself, Birgitta Cappelen and Fredrik Olofsson at NIME 2014. The title of their paper is “Designing Sound for Recreation and Well-Being”. NIME, New Interfaces for Musical Expression, is the most important conference and community within music and technology. RHYME has for several years now been one of the few projects in this community focusing on developing musical artefacts and expression in a health related context. This is the third time RHYME has published at the NIME conference. NIME 2014 was held at Goldsmiths College at University of London, 30 June – 4 July 2014.
On 29th of June RHYME member Anders-Petter Andersson held a presentation on the ESSA 2014 Conference. ESSA is the European Sound Studies Association annual conference. This time the Conference name was Sound Studies: Mapping the Field and was held in Copenhagen from 27th to 29 of June 2014.
ESSA is a community that has emerged from various traditional disciplines within the humanities and social sciences related to sound and music, such as musicology, art history, media and cultural studies, psychology, architecture and urban planning. Sound studies has been associated with inter- and trans-disciplinary approaches.
Anders-Petter Andersson presentation was titled “Creating Musical Actors” and is about the musical work in RHYME and Polly, the 4th generation of Co-creative Tangibles.
In his presentation Andersson argues that music is not longer only the sound object, instrument or piece of art, but the services around it. These are services in the form of actors with characters that behave coherent as they answer back musically, take initiative and develop over time. With perspectives from drama, game theory, actor-network-theory as well as from learning theories on motivation, Andersson explores two design cases from RHYME.
After a long and very demanding design and development period of the 4th generation of the Co-creative Tangibles, named Polly World, we have conducted a first pilot study. Polly World is an attempt to answer all the demands, suggestions and wishes from users and experts related to the three earlier generations of RHYME’s Co-creative Tangibles, in addition to Social Media functionality. Developing the Polly World has therefore represented much more work and complex challenges than we initially planned for and understood. It has therefore extended the project massively. Some of the major challenges and user demands have been:
- Dynamic and interactive change of microphone input in mobile Co-creative Tangibles. So the users continuously can change their singing while interacting with diverse sensors of the wireless Tangibles.
- Closer and more intimate and embodied relation to the video projection, compared to traditional wall projection, TV and Computer Screens.
- Better sound quality in the mobile Tangibles, regarding both sound frequency range and volume.
- More varied musical choices and tunes ( for expanded the self-regulating possibilities), regarding forms of musical interaction (musicking), type of music and sound experiences (music tunes, genres and soundscapes), so the users can easily handle and choose music, music related activity and intensity level.
- More robust transparent textile surfaces, since light is an important response dimension both in the interactive surfaces and part of the whole audio-visual experience.
- Better touch sensor solutions, both regarding interaction possibilities and surface qualities like colour, softness, light response, responsiveness and durability.
- More engaging and sensorial stimulating surfaces, to engage and motivate interaction with non-computer surfaces to gain a more fluent tactile, sensorial and interactive (computer based ) experience.
- Easier grips and more possibilities to handle input sensors like microphone and bend sensors.
- Possibility for the user to include their own things into the musical experience and make their own things sing along and expand the self-regulating possibilities.
- Easier battery charging of the mobile Tangibles, so that the users easily can do it themselves.
- Easier start and stop of the system, so the user can handle it themselves.
- Movable Tangibles, so the whole Polly World can be moved, installed and stored rather easy. These practicalities took days and were in several cases impossible with earlier generations of the Co-creative Tangibles…
- Social Media functionality for distributed interaction on Smartphones and Tablets over the Internet (will be described in detail in a later post).
All these user and expert wishes (and many more) we have tried to put together in the new generation of the Co-creative Tangibles, Polly World. The Polly World consists currently of 4 Interactive, networked things, Co-creative Tangibles. The biggest , Polly Land, is wired , and 3 are mobile and wireless. Polly Land contain a close projection using back ground projection onto a acrylic surface included in the soft textile landscape. Polly Land has three “arms” containing RFID-reader, camera and microphone. It has a stand alone Wi-Fi system. The three other Polly interactive things is Polly Planet, Polly Fire and Polly Ocean.
They have their own separate colour so they are easy to identify and interact with using a graphical interface on Smartphone and iPad. But the different sensors has the same shape, surface and interaction form. For instance is all touch sensors soft velvet triangles with the same output light pattern, but in Polly Land they are blue, in Ocean they are turquoise, in Fire they are orange and in Polly Planet they are yellow. The same is the case with all input sensors like the bend sensors, the microphones, the RFID-readers and inductive charger.
The different shapes are built on traditional shapes within the field, such as Polly Planet that is bases on a ball shape. The shapes are constructed of triangles joined together with strong bands like trails the users can follow around the shapes to find more sensorial challenges and experiences. The input areas are coloured sports fabric that contrasts the passive grey wool patches. Every Polly Tangibles, interactive thing has big light output patches included, some with speakers embedded.
The RHYME development team are currently working on creating the music for the 4th generation of RHYME Co-creative Tangibles. The goal is to meet the demands from the users, music therapists and focus groups, to choose the music of their own liking (self-regulation), both well known music and more abstract soundscapes. We edit and de-compose a linear tune into musically significant elements. Elements that the users can re-compose, based on advanced musical rules into a continuous musical experience, by interacting together. The different users, both at home and on a distance, can interact and co-create the music together. The co-creation can be achieved in many ways: through physical interaction in the Co-creative Tangibles; through a graphical iPad interface; or through writing compositions in a high level musical Twitter language on the mobile phone.
The 4th generation Co-creative Tangibles, named Polly, will offer the users a broad range of musical styles from beat-based popular music, children songs, classical music to more contemporary musical experiences. Currently (June 2014) we have about 40 different interactive visual and musical soundscapes the users can select from by choosing preferred Scene Cards with RFID tag. Under are some of the Scene Cards Polly offers so far, but this part of Polly is in continues development, so we can offer new visual and musical experiences :