The technology was brought to us by Kate Stone and her team at Novalia in London. We had our partners at DCC print the posters on Accent paper. We perfected the color and everything to make sure it was just right, and we shipped those posters to London. That’s when Novalia built the circuitry into the posters.

There’s circuitry between the Accent paper and the foamcore backing. On the back of the foamcore, there’s a power source. In this case we used batteries so they could hang by themselves with no visible wires.

The synesthesia idea was brought to us by our client because they had a friend who had it. Synesthesia is basically seeing colors when hearing music, and hearing sounds when seeing colors. We thought if the printed piece was the source of synesthesia, then it’s almost an instrument. We reversed it so the colors were making the sounds. We thought about what that could mean. Could it be a rainbow? Could it be a spectrum? In the end, we landed on the color wheel. The audience at the conference was young designers, so we wanted to speak in a language that young designers would understand.

We approached it by asking,“If a 2D piece could speak to you, what would it do?”. In thinking about posters, we really wanted to push that. We came up with a bunch of different ideas – at least 15 of them. We had simple things. For instance, when you see all of the different images in the windows of a building and wonder, “What’s going on in those buildings?”. One concept we had was being able to touch every window in the building, so we had someone singing in front of the mirror with a hairbrush and someone watching TV. We had a lot of different things going on in this apartment building. We went back to things such as mysteries. The idea of radio mysteries.

We had to be able to fulfill the requirements. You’re not just creating a poster, you’re creating sounds. So you have to fulfill the need for sound. So how do you make these sounds? Do you need actors; where do you go? Every idea we came up with, we had to be able to deliver on the audio too. Another idea we had was murder mysteries. We created a great illustration of a building where the a crime had taken place. Like the old Dell mystery novels that would show the place of the crime before you would read it. So we wanted to show the crime and, as you touched different parts of the poster, you could solve the crime and understand what happened.

‍

Winston, our content strategist, is a composer and a musician. He composed the music that you could play with the color wheel. He figured out how to make the color wheel an instrument. Then we had other ideas as well. Onomatopoeia are words that are created to represent sounds. They have no other purpose than to represent sound. “Gurgle, splat, sizzle.” We represented these types of sounds graphically. When you touched them, you could play them and mix all of the different sounds.

The team at Novalia does a lot with music. Particularly with mixing music and festivals. We knew that a lot of beats are pulled out of music from the 60s, 70s and 80s, so we took images from those decades. We found imagery, like Michael Jackson’s glove, a Ford Pinto, a lava lamp – all of these different things tied to different decades. Winston then found casio music software, which was one of the first widespread sampling keyboards. He loaded it on his computer and made all of the sounds and beats for the different imagery. People at the conference could mix music with the poster.

‍

It’s interesting because it goes forwards and backwards. Before this technology existed, you could never do it. You’ve got the extreme, cutting-edge technology, but then you also have the physical experience of touching things and making things happen. That goes all of the way back to carnivals, museums or anywhere where you go that’s interactive and there’s a cause and effect that excites you. You have to look at both worlds. You have to look at the physical experience, which is timeless, while also looking at what that technology allows you to do. We considered all of them and really tried to create something that would engage people on a cerebral level, and also on a physical level. So that was really our approach. To really open everything up. We were playing into ideas that were timeless.

‍

Even though we use computers so much, we think we absorb everything in more of a cerebral way. We knew that the mind likes to learn physically from our experience working with Posit Science and spending so much time with neuroscientists learning about brain plasticity. In fact, we know that physical learning combined with mental learning is really challenging for the body. Such as, learning to dance. That’s why as you’re getting older, and your brain is aging, doing things like learning a new language can help. But even more so, learning new dances are even better for your brain because it really challenges your mind. Think about it, when you’re learning a dance, you feel like an idiot. You feel like you have no coordination and can’t do anything because that’s how challenging it is for the brain.

We wanted to make sure that we had concepts that dealt with both physical and mental learning. That’s why we had ideas like synthesia. It’s as much cerebral as it is physical. You’re being physical while trying to play the music, and it was quite a challenge. Like mixing music is just sort of fun. You hit it and it happens. But as you do it, you start to learn how it works. So that engages you in one way. The onomatopoeias, which seem really simple, had sound quality that was so jarring and amazing for people. Hitting those even had an element of surprise. So we tried to incorporate that on all levels. With things like the synthesia and the design process, we wanted to drive it a little bit more cerebrally. We wanted to engage people physical and mentally on all levels.

‍

I think it does because you have the element of surprise. A printed piece, a book – everyone is really familiar with the interface of a book or even a poster. You look at it, turn the pages and know what’s coming. When you’re doing something interactive like this, there’s the element of surprise which is really wonderful. There’s also cause and effect — because you’re causing things to happen, and no two people experience it the same way. Depending on what you touch or do, it will change the experience and it creates a unique interaction every time that is one of a kind. There’s certainly no other way that you can do that. Except through interactivity in a device, but even then, that’s not the same as being able to do this in a physical environment where you’re touching a piece of paper.

‍

We wanted to speak to the younger audience about being a designer. The idea that it’s very emotional to go through the design process. Every time you go through it, you never feel like you’re going to be able to solve the problem. It happens to me and it doesn’t matter how old you are – it happens all of the time. At first, you’re overwhelmed and you might be confused. Then, there’s usually a spot in the process where you’re like, “Oh my god, I’m never going to figure it out”. Suddenly you hit another part where you do start to figure it out and you start to gain confidence until you get all of the way to the end where you’ve solved it. Then you’re feeling super confident saying, “I knew this all along and of course I can figure it out.” It’s really an emotional rollercoaster. We wanted to create that, and we thought, “How do we explain that to the designers? How do you do it visually?” It was really a struggle. Then one night, Lindsey and I were texting back and forth trying to think through what we could do. We came up with the idea to use emojis.

We used emojis for two reasons. First, it appeals to a younger audience and it’s a product of the millennial generation. Secondly, emojis don’t speak. Wouldn’t it be a fun idea to flip it on its head and have emojis represent the different stages of the design process. From curiosity to being overwhelmed all of the time and all of the way to jubilation. We created emojis that represented each phase. As you go through the design process, you can tell what the feeling associated with it was based on the look on the emoji’s face. We showed the entire process. From how overwhelming it can be to all of the way to the end where the emoji is saying, “I knew it all along and I’m a genius.” I ended up becoming the voice for the emojis. We actually wanted to create a genius button that designers could hit every time they solved a problem.

When we finish doing concepts, we always have ourselves do one more push to see if we can go further. Knowing what we know now, what would we do? What would we do if the Museum of Modern Art was doing a whole exhibit on interactive posters like this. How would they push the concept? We wanted to go even further so we thought, “If you really push it this way, and a poster could speak to you, then the poster wouldn’t need imagery at all”. It could tell you, and you could see it in your mind’s eye. Then we thought, “What if interviewed like 50 people from different cultures and got them to tell us the most amazing thing they’ve ever seen in their lives and have them describe it?”. Mine would probably be when I stumbled upon the aurora borealis in Canada.

We knew that we could store up to 50 different samples within one poster, and we didn’t want people backing up at the exhibit. Each person would be able to put their hand up and we would have a random delivery of a different story. We would put a hand shape on the poster, and then you would have someone telling you the most amazing thing they’ve ever seen in their life. Therefore, the poster creates the visuals in your mind’s eye. You no longer even need to see something visual because the poster is communicating the visual in your mind and allowing you to see it yourself. And each person would see it differently because each person has had unique life experiences.