On 3rd July RHYME member Anders-Petter Andersson made a presentation of a paper by himself, Birgitta Cappelen and Fredrik Olofsson at NIME 2014. The title of their paper is “Designing Sound for Recreation and Well-Being”. NIME, New Interfaces for Musical Expression, is the most important conference and community within music and technology. RHYME has for several years now been one of the few projects in this community focusing on developing musical artefacts and expression in a health related context. This is the third time RHYME has published at the NIME conference. NIME 2014 was held at Goldsmiths College at University of London, 30 June – 4 July 2014.
On 29th of June RHYME member Anders-Petter Andersson held a presentation on the ESSA 2014 Conference. ESSA is the European Sound Studies Association annual conference. This time the Conference name was Sound Studies: Mapping the Field and was held in Copenhagen from 27th to 29 of June 2014.
ESSA is a community that has emerged from various traditional disciplines within the humanities and social sciences related to sound and music, such as musicology, art history, media and cultural studies, psychology, architecture and urban planning. Sound studies has been associated with inter- and trans-disciplinary approaches.
Anders-Petter Andersson presentation was titled “Creating Musical Actors” and is about the musical work in RHYME and Polly, the 4th generation of Co-creative Tangibles.
In his presentation Andersson argues that music is not longer only the sound object, instrument or piece of art, but the services around it. These are services in the form of actors with characters that behave coherent as they answer back musically, take initiative and develop over time. With perspectives from drama, game theory, actor-network-theory as well as from learning theories on motivation, Andersson explores two design cases from RHYME.
After a long and very demanding design and development period of the 4th generation of the Co-creative Tangibles, named Polly World, we have conducted a first pilot study. Polly World is an attempt to answer all the demands, suggestions and wishes from users and experts related to the three earlier generations of RHYME’s Co-creative Tangibles, in addition to Social Media functionality. Developing the Polly World has therefore represented much more work and complex challenges than we initially planned for and understood. It has therefore extended the project massively. Some of the major challenges and user demands have been:
- Dynamic and interactive change of microphone input in mobile Co-creative Tangibles. So the users continuously can change their singing while interacting with diverse sensors of the wireless Tangibles.
- Closer and more intimate and embodied relation to the video projection, compared to traditional wall projection, TV and Computer Screens.
- Better sound quality in the mobile Tangibles, regarding both sound frequency range and volume.
- More varied musical choices and tunes, regarding forms of musical interaction (musicking), type of music and sound experiences (music tunes, genres and soundscapes), so the users can easily handle and choose music, music related activity and intensity level.
- More robust transparent textile surfaces, since light is an important response dimension both in the interactive surfaces and part of the whole audio-visual experience.
- Better touch sensor solutions, both regarding interaction possibilities and surface qualities like colour, softness, light response, responsiveness and durability.
- More engaging and sensorial stimulating surfaces, to engage and motivate interaction with non-computer surfaces to gain a more fluent tactile, sensorial and interactive (computer based ) experience.
- Easier grips and more possibilities to handle input sensors like microphone and bend sensors.
- Possibility for the user to include their own things into the musical experience and make their own things sing along.
- Easier battery charging of the mobile Tangibles, so that the users easily can do it themselves.
- Easier start and stop of the system, so the user can handle it themselves.
- Movable Tangibles, so the whole Polly World can be moved, installed and stored rather easy. These practicalities took days and were in several cases impossible with earlier generations of the Co-creative Tangibles…
- Social Media functionality for distributed interaction on Smartphones and Tablets over the Internet (will be described in detail in a later post).
All these user and expert wishes (and many more) we have tried to put together in the new generation of the Co-creative Tangibles, Polly World. The Polly World consists currently of 4 Interactive, networked things, Co-creative Tangibles. The biggest , Polly Land, is wired , and 3 are mobile and wireless. Polly Land contain a close projection using back ground projection onto a acrylic surface included in the soft textile landscape. Polly Land has three “arms” containing RFID-reader, camera and microphone. It has a stand alone Wi-Fi system. The three other Polly interactive things is Polly Planet, Polly Fire and Polly Ocean.
They have their own separate colour so they are easy to identify and interact with using a graphical interface on Smartphone and iPad. But the different sensors has the same shape, surface and interaction form. For instance is all touch sensors soft velvet triangles with the same output light pattern, but in Polly Land they are blue, in Ocean they are turquoise, in Fire they are orange and in Polly Planet they are yellow. The same is the case with all input sensors like the bend sensors, the microphones, the RFID-readers and inductive charger.
The different shapes are built on traditional shapes within the field, such as Polly Planet that is bases on a ball shape. The shapes are constructed of triangles joined together with strong bands like trails the users can follow around the shapes to find more sensorial challenges and experiences. The input areas are coloured sports fabric that contrasts the passive grey wool patches. Every Polly Tangibles, interactive thing has big light output patches included, some with speakers embedded.
The RHYME development team are currently working on creating the music for the 4th generation of RHYME Co-creative Tangibles. The goal is to meet the demands from the users, music therapists and focus groups, to choose the music of their own liking (self-regulation), both well known music and more abstract soundscapes. We edit and de-compose a linear tune into musically significant elements. Elements that the users can re-compose, based on advanced musical rules into a continuous musical experience, by interacting together. The different users, both at home and on a distance, can interact and co-create the music together. The co-creation can be achieved in many ways: through physical interaction in the Co-creative Tangibles; through a graphical iPad interface; or through writing compositions in a high level musical Twitter language on the mobile phone.
The 4th generation Co-creative Tangibles, named Polly, will offer the users a broad range of musical styles from beat-based popular music, children songs, classical music to more contemporary musical experiences. Currently (June 2014) we have about 40 different interactive visual and musical soundscapes the users can select from by choosing preferred Scene Cards with RFID tag. Under are some of the Scene Cards Polly offers so far, but this part of Polly is in continues development, so we can offer new visual and musical experiences :
RHYME member Birgitta Cappelen was invited to speak on this year’s ArcInTex conference in Gothenburg. The title of her lecture was “Designing Component Based Smart Textile” and concerned the ongoing work with developing textile solutions for 4th Generation of Co-creative Tangibles, named Polly. The idea is to combine perspectives from the Textile and Computational traditions into a new concept of Smart Textile. From the Textile tradition we have modular thinking in Patchwork were we reuse used materials into creating new artefacts. In our Folk Costume tradition we also have modular parts like the belt and chest part (in Norwegian “bringeduk“) which can be added, inherited and used in another costume. Development in Carpet Tile Industry, from being a material to a service provider, is also an important inspiration. In the Computational tradition we have the tradition of modularity and components in both hardware components and in different forms of Component based software architecture. In developing RHYME Co-creative tangibles, Polly, we use both these lines of thoughts to structure the Smart Textile vertically and horizontally. Horizontally as different surface areas of sensors and sensorially stimulating surfaces. Vertically as layers, from the textile surface and shape, via hardware components, different functions and services to the data protocols (USB, Twitter, TCP/IP ).
On November 9th the RHYME member Anders-Petter Andersson was invited to give a keynote lecture at the 3rd Music for All Conference in Helsinki. His lecture was called “Why Electronic Music Technologies Collect Dust, People Get Angry with Fury Robot Seals and a Possible Solution”. The conference was given in collaboration between the Resonaari Special Music Centre, the Universities of Helsinki and Jyväskylä.
Currently we are working intensively with the development of the 4th generation of Co-creative Tangibles. The 4th generation of Co-creative Tangibles are in the project plan defined as “distributed and social media”. In the RHYME project this means that the users can co-create across different media, space and time. For instance that a sibling can interact through a smartphone or tablet from another place, with her sister or brother and the Co-creative Tangibles at home.
In the 4th generation we try to accomplish, develop and realize as many of the ideas, suggestions and wishes as possible, that we have received from the users, care persons, families and experts from several fields throughout the whole project. These many ideas, suggestions and wishes have been articulated through participation in the many workshops, tests, interviews, discussions and observations during the whole RHYME project.
Some of the users’ wishes, such as dynamic microphone input to the music together with different forms of sensorial stimulating input sensors in a mobile interactive thing, has been a major technological challenge. These user wishes have compelled us to develop six different hardware platforms throughout the project, which was not something we planned to do, but had to do because of its importance for the users and music therapists.
Including direct light response and projection in the tangible objects has also been an important user demand that has been a great design challenge. We try to include and prioritize as many user wishes as possible into a new 4th generation of distributed and social co-creative tangibles with new shapes, structures, surfaces, hardware, software, music, set design, positive experiences and mastery levels, personalization and interaction forms. Here are some pictures from the ongoing development process:
New materials, surfaces and joinery under development:
During the 9th European Music Therapy Congress in Oslo, RHYME was presented in several ways. RHYME member Even Ruud, Karette Stensæth and Anders-Petter Andersson contributed to the conference. Even Ruud started the conference with the keynote “Music – A cultural Immunogen”. On Thursday 8th August Anders-Petter Andersson presented his and Birgitta Cappelen’s paper Vocal and Tangible Technology for Music and Health and Karette Stensæth was chair and released her paper “Musical co-creation’? Exploring health-promoting potentials on the use of musical and interactive tangibles for families with children with disabilities published in International Journal of Qualitative Studies on Health and Well-being.
Yesterday NRK (Norwegian Broadcasting Corporation), both radio and television came and made interviews with parents contributing as users and research persons in the project and RHYME’s design team. Today at 8.04 am. the radio reportage was sent on NRK P2 Kulturnytt.
The RHYME project was presented in a inspiring article under the title “Music furnitures helps children with special needs” at the norwegian research news site forskning.no. In the article both parent Inga Bostad and RHYME project member Birgitta Cappelen was interviewed. Inga Bostad experience that her daughter has become more curious and willing to explore new things after participating in the RHYME project. Read the article (in norwegian) here.