PanoMoments Unity Toolset, A New Format for Animators in VR: Conversation with Dustin Kerstein / by HIGHWAY101

PanoMoments are a new short-form VR medium that brings together time and space. I recently met with PanoMoments founder Dustin Kerstein to talk about their recently released Unity3D Toolset, which makes it very easy for VR/360 creators to convert their 360 video and/or animated scenes into PanoMoments. You can see an example here built from a scene of the 360 experience Kinoscope.

Alex: What are PanoMoments? Are they animations, 360 videos, the gifs of VR?

Dustin: I’m very open calling PanoMoments the gifs of VR. I ended up moving away from calling them 360 or VR. I call them living photos now. They’re impossible to describe and I don’t necessarily think of them as being limited toVR. That’s why I’ve embedded a PanoMoment as the banner of our website. PanoMoments are more engaging than a regular banner photo, and for that reason they will be a really great bridge into VR for people getting out of the 2D web into WebVR.

PanoMoments as the gif of VR? I’m ok with that. That’s a natural way to see it but if I market it like that, people will not think of it in the right way. I like to see it a little bit like the VR game ‘Superhot’. I love this concept of playing with space and time, nobody moves when you don’t move.

A: What a shift, from creating a physical rotating camera to building a virtual camera in Unity3D! What happened? Tell me more.

D: I’ve been working on PanoMoments for a year and a half now. I’ve been mainly focused on the physical rotating camera method. But it just isn’t very practical as it requires extra hardware. I thought about creating a virtual rotating camera but I didn’t realize how easy it would be to build it within Unity3D. Actually, that wasn’t possible until early 2018 as only Unity3D 2018.1 beta allows 360 degrees captures without any plugins. I had to take advantage of it. I finally spent a day putting together an ugly, terrible Unity sample project but at least it’s functional and simple enough for people to start with! It is a very simple scene where a camera is capturing the 360 environment with a virtual rotation. I’d like to make it available as a plugin that other developers could integrate into programs like Tilt Brush. I’d love for VR animators to be able to export directly from VR into PanoMoments. To me, it’s a natural medium for animators. It’s like a flipbook.

I want nothing more than to find a way to fire up the creative community and get the creators to build things within this format.

Alex: Did you already initiate conversation with any VR/360 animators? What are you looking for from the VR/360 community?

D: The very first step was getting out the demo project and post some instructions online. Now that it’s done, I need some better content. I reached out to a couple of animators including Michael Lippy. He’s based out here in CA and has done a little bit of 360/VR animation work but he’s mainly a traditional 2D animator. I pitched him PanoMoments and he didn’t immediately see how it would apply it to his animations. It took a little bit of talking to get him spinning around some ideas. The one that really triggered him was the multiple rotations feature, which actually isn’t supported just yet. For now, I like to think in terms of composites. For example within one loop you can composite three different scenes. You don’t have to stay within one single environment. It’s a simple and interesting trick. Creating PanoMoments certainly requires a little bit of outside of the box thinking. At this point, I really need to understand what the hesitations are from the illustrators’ and animators’ perspective. My first gut feeling was “Yeah, let’s give this tool to animators and build stuff!” Yet, I didn’t get that reaction when I first tried to explain it via emails. Maybe the content that I was showing as examples didn’t make them click. I don’t know. I need to understand how to get people to immediately see the possibilities behind PanoMoments, and what a good workflow for animation looks like, as well as what could possibly be achieved within this format.

A: What are the next steps regarding the Unity Toolset?

D: The demo project is an offline rendering tool. It’s very slow to export high quality photos. This has not been designed for real time. Doing real time is more difficult but doable. Plugins like Rock VR, and even Unity3D technically allows it. However it’s not nearly as polished as the Rock VR solution. I think our next step for the Unity Sample Project is to synchronize the rotation of the camera with the animation itself. Right now, in the demo scene, the animation is doing its own thing and the camera rotation is independent from it. This has to be addressed. If we really want to build perfect loops, and seamless animations, we need to make sure that the animation is synchronized with the rotation. That’s something that should be easy to do in Unity. I’d love to update the instructions for how to do that because that’s going to be a very important piece of building content.

Unity Example Scene PanoMoments Unity3D Toolset.

Unity Example Scene PanoMoments Unity3D Toolset.

Quick tutorial PanoMoments Unity3D Sample Project:

A: What about bringing soundscapes to PanoMoments?

D: If that’s really the thing that people want, I would do that in a couple days. Adding sound is something I’ve been wanting to do for a very long time. There are two ways of doing it. 1-Synchronizing sound to the timeline. This is technically more challenging to implement and it might not sound particularly good as we would also hear audio clips going backwards. A lot of things won’t sound good backwards.! But somethings might be really cool and I’d love to make that an option for people that want to experiment with it. 2-Adding background audio or music. That’s really simple and I can do that in a couple days. If that’s how the creative community wants to use PanoMoments, I would drop everything and do that. More than anything, I need to find that community.There’s been good response on the viewing side, but the creation process has been too difficult until this point. And while anyone can now easily convert 360 videos into PanoMoments, I haven’t really pushed that feature yet.

D: I’m trying to figure out how to best approach the virtual reality animation community. It’s my number one priority right now. I’d like to show that PanoMoment are not just a gif, but something more. I truly think that we can tell stories with PanoMoments, a go at your own pace experience. I think animations are probably the best ways to showcase the possibilities of the format. It might be a challenge to build. It will require some digging into the format, and understanding how to work with it.Most of my captures have been limited to simple quick rotations or time lapses. Things start to get much more interesting when you’re willing to actually produce or composite scenes together, like the example on our homepage which is a composite of three PanoMoments. That technique can be exploited in animations in a bunch of different ways. I think that’s really where we’re going to see emerging new creative content, as well as stretching the boundaries of what is possible with the format. Eventually, it might be the photo advertisement of VR. This format has a serious potential for building marketing and advertising commercials. The easiest way to show it, I think, is through animation. I’ve been talking to a couple of studios in LA. One of them wanted to film a beer commercial or Rune Goldberg scene using the PanoMoments format. But it would take at least $30K to produce it. Building something that emulates that kind of experience through animation totally changes the equation, as well as the creative possibilities.

A: Awesome, thank you Dustin.

PanoMoments needs more example scenes for the release of their next update of the Unity Toolset Sample project, as well as adding a raycast script to the camera so it synchronizes frames with the camera rotation. If anyone feels like that is something they can do, please manifest yourself. Contact me at alex at