Whisperer

👻🪴 A cozy open source VR demo for Meta Reality Labs that showcases voice interactions using the VoiceSDK and Wit.ai, through an immersive ghost story in virtual reality.

In Whisperer, you play a ghost trying to transcend the mortal plane. To do so, you must help our heroine, Bea, rekindle her love of gardening. 🌻 The only tools at your disposal are your voice, your wits, and scattered hints from a bird named Harold! 🦜

Production Style

When I think back on Meta Whisperer, one of the main things that sticks out is how great it was to work with Eric Chang. I think this was his first VR or interactive spatial project, and his curiosity and open-mindedness was felt from the very start.

The interaction design, theme, art style, and technical implementation all came together beautifully during a production that was flexible enough to allow everyone’s best ideas to rise to the top.

The result was a project where everyone’s contributions made it better than it should have been – from the cute characters to the wonderful setting with multiple lighting setups, novel interaction system and more.

Novel Interaction System

We developed a novel interaction system where, in one action, the user both selects an object and activates the microphone to listen for voice commands that get piped to Wit.ai – all while keeping within the story’s theme of being a ghostly apparition.

Your hands act as a reticle to select objects you’re then able to command with your voice. This makes for an intuitive metaphor, free of extraneous UI elements, that first-time users can pick up with ease.

Shout out to Anthony Enns for nailing the feel of the raycasting system.

A Lovely Set

When determining the scale of the environment, we landed on a sweet spot that had plenty of charm and versatility, but didn’t require any player locomotion. This let us focus on player interactions, character animation, the feel of the environment, and the Wit.ai tech that the project is meant to showcase.

The cozy greenhouse where the story takes place

The result was a little greenhouse environment that also served almost as a stage for a play. With different lighting environments by Thomas Robertson, testing in Bea’s greenhouse was such a joy!

My Contributions

I handled Wit.ai intents setup and training, as well as interaction design and the UI messaging system. I implemented animations, interactable props, and physics effects throughout the scene—managing all the props you can interact with using your ghostly powers.

I also scripted Harold the Bird (nothing fancy, but he’ll talk to you!) and wrote the GitHub documentation. There was probably some onboarding scripting and other things. Again, this one was hugely collaborative.

Open Source & Accessible

What started as an example project for developers to learn from turned out so well that it ended up on the Quest Store. In addition, all code and assets are accessible and 100% open source on GitHub!

Credits.