We love stories. People have stories, places have stories, even something as arcane to an average person as agile software development relies on so called user stories. That’s because stories are how we relate: to people, to places, even software products.
Hellicar&Lewis, a London-based design studio, wanted to make stories more interactive with Estimote Stickers. Their project, Sound Clips, uses technology to help children with special educational needs develop their imagination through storytelling. Below you can read how the project came to be.
Mighty Mega stories
Discover - Children’s Story Centre explores how storytelling can, in their own words, help improve reading, writing, speaking and listening skills for children in fun and imaginative ways. On weekends, they organize Mighty Mega Club: an event for children with disabilities and special educational needs. The kids participate in storytelling sessions, where adult story builders help them craft stories using abstract physical props. This year, Discover wanted to push further into interactive experiences. They hired Hellicar&Lewis to help Mighty Mega Club figure out new ways to use technology to inspire children.
The first step H&L took was to observe the sessions: The problem was that we didn’t want to design a product that took children’s attention away from the props themselves, or interrupted their creative process with lots of technology. We wanted something that was instant, rewarding and lightweight physically and conceptually. said Joel Lewis, Co-Founder and Interaction Director at Hellicar&Lewis.
They settled on sound.
Chirping nearables
H&L wanted to let children and story builders enhance their stories with sounds. A bird chirping, a door creaking, a person giggling: little touches that ignite the imagination. But how to do that in a way that doesn’t feel artificial but seems like a natural extension of storytelling?
The answer is: Estimote Stickers. H&L developed an iPad app with tens of sound effects, grouped into nine categories: Domestic, Farmyard, Jungle, Ocean, Silly, Space, Transport, Weather, and new user recordings. Every clip is tied to an Estimote Sticker hidden in a custom enclosure. Nearables activate the app when moved and iPad responds with a sound. It happens instantly so the story’s narrative is never disrupted: only enhanced by the sound effects.
Estimote Stickers allowed users to concentrate on making stories and think about the sound effects that should accompany them, rather than being distracted by unnecessary technology. Ultimately we wanted to augment existing behaviour rather than trying to change the already amazing process of Story Building. adds Lewis.
Overcoming hurdles
Development started by using a technique known as Paper Prototyping to build a prototype of the product with the team at Discover before writing any code. After establishing how and why the project would function with the client, H&L were then ready to translate their prototype into a working digital/physical product.
H&L relies on openFrameworks for software development so their developer, James Bentley wrote a wrapper around Estimote iOS SDK and used the the ESTNearableManager object for managing interactions with nearables. When they moved on to testing the first prototype, a new issue appeared - the accelerometer in Estimote Stickers was too sensitive, triggering sound clips even at the slightest movement. It caused the iPad to play multiple sounds one after another, leading to cacophony instead of immersion. So H&L kept testing and tweaking the interaction mechanism. The benchmark of success? When the banana felt right (you will need to watch the video above to understand what they mean by that).
With the software side done (it’s open source and you can access it on GitHub, H&L could start working on the physical side of Sound Clips. Estimote Stickers were too tiny for the Mighty Mega sessions. H&L also wanted something more resistant to squeezing, pushing, and all the wear and tear inevitable for toys.
They thought about 3D printing but after a while moved on to felt pouches. They’re nice to the touch, durable, and affordable: a perfect combination. Every pouch also comes with a clip so they can be attached to any prop to allow for fun combinations. For example, have you ever heard a banana sing like a bird?
We felt the pouch design worked nicely with our overall “ABC" Aesthetic. The pouches provided a nice literal touch point between the app and the sound clips as well as creating a tactile interface that felt right with the Discovery Centre’s prop based analogue story telling approach. It’s also important that the clips themselves could pretty much attach to anything the kids and storytellers might use - cloth, wooden spoons, hats, bananas, paper, dolls etc. The bulldog clips were the first things we used in testing and as the old saying goes “if it ain’t broke….” The nice thing about the bulldog clips is that they are cheap and replaceable and available on every high street - something that isn’t true for 3D printed custom parts! explained Peter Hellicar, Co-Founder and Creative Director at Hellicar&Lewis.
In conclusion
The video at the beginning of this post says more than a 1,000 words about the quality of Sound Clips. H&L did incredible work implementing nearables as a storytelling tool. When you see your product used to help children foster their imagination, you can only be proud. Thanks H&L for making us so proud!
Wojtek Borowicz, Community Evangelist at Estimote