Do-ems!
10 tiny playful collaborative experiments!

Join us in the dome for some tiny game-poems! We’ve been exploring collaborative control and cooperative play. This is research-in-progress.

=> Kali: Draw together on the dome, kaleidoscope style.
=> With Me: Eat dots to grow big and collide to regrow the world.
=> Stick Together: Stay within a moving sphere together.
=> Hug (Knus): Match faces to find connection.
=> Flock: Herd your little friends around.
=> Steer: Collectively steer the dome.
=> Share: Control a large sphere together, growing or shrinking.
=> Stars: Match the constellations.

Project Overview:

I made 10 tiny collaborative game-poems for the Satosphère (SAT) a massive 13-meter-wide dome with 8 projectors and 93 speakers that can hold 350 people. That said, I’ll probably limit the games to 10 people at a time (at most) for now!

We make the 10 game-poems interactive by tracking people’s movements with a camera. This way you can make things happen by just moving your body. We climb up into the scaffolding and pop the top off the dome to mount the camera (MQ042RG-CM) 13 metres up in the air. Of course, a camera doesn’t actually tell us positions of people in numbers (it’s just an image, not data), so we use LivePose, an open-source computer vision library, with old-school blob tracking to determine where people are and where they are moving.

LivePose continuously sends messages about people’s positions using Open Sound Control (OSC) to Unity 2022.3.29f1.

In Unity, we create an avatar for each person (ID) that we receive. This lets each person control a player in the games! The system is a bit glitchy as the camera often loses track of players or mixes up their IDs. We dream of LIDAR.

Because tracking people is difficult, I'm making headbands with infrared LEDs (940nm wavelength, 1.35V, 20mA), powered by 3V CR2032 coin cell batteries with 82-ohm resistors. We hope this will make the camera better at detecting players. It’s hard to play with anything when your controller keeps fading in and out of existence!

In Unity, I'm always listening for OSC messages from LivePose. When we get a message, the message includes a player ID and an X and Y position. In Unity, we convert those 2D XY coordinates into a curved space (calculating a realistic Z or up/down dimension) so we can put things in the right spot in a 3D Unity world. We also manipulate these coordinates a bit to save us from neck pain. Traditionally, dome shows are viewed reclining on beanbags. So we squash everything closer to the horizon, so we don’t have to look straight up.

In Unity we also handle all the game logic. Each game is in its own scene. We reuse a lot of the code for OSC handling and creating player avatars, with a C# script called “MainController” common to all the scenes.

For rendering, we use FullDomeForUnity, which simulates the fisheye lens needed to project our Unity world onto the dome. I initially thought projecting a 3D world onto a dome would be straightforward—I was wrong.

Every time a sound is triggered in Unity or something that is playing a sound needs to move spatially (93 speakers!), we send an OSC message to TouchDesigner using extOSC. We use Ableton to play the sounds and SpatGRIS to spatialize them in the dome.

SpatGRIS moves sounds across the 93 speakers using OSC messages containing azimuth, elevation, and radius. Ableton plays the sounds, while TouchDesigner bridges the gap between Unity and Ableton.

Thanks:

Special thanks to the wonderful people who helped bring this project to life: Isaac (beauty), Pen (love), Manu, Jules, and our consultants Pierre, Celine, Julia, and Alexis. A big thanks to the incredible team at SAT: Marek, Kristophe, Bruno, Mourad, Hugo, Zack and other Zack, Pablo, OSSIA, and Jean-Michaël. Our heartfelt gratitude to our play testers: Julia, Lee, Tricia, Don, Libby, Kousu, Isa, and Emma, for their time and feedback. Thank you to Fili for music and sounds, Kristian North, Jonsi, and Alex Somers for the music.