A dream came true last week-end at Montreal's Hacking Health hackathon: our team hacked a VR headset to make it EEG/ECG fully equipped; a first class equipment blending Vrvana's headset -the Totem- and OpenBCI's.
Since a long time at the University of South California, Dr A. Rizzo works at the intersection of VR and brain imaging. I can't remember how many times I hear people/websites quoting him: "I have no question that Oculus will revolutionize virtual reality for clinical purposes." Early 2015, I experimented with the Dk2 and the EPOC headset and, despite the fact that it is a mission impossible to wear the rift and the EPOC, I've been amazed by the potential of such acombination. In march 2015, Mindmaze created a surprise announcing a thought-powered virtual reality headset: MindLeap. But we didn't hear of them since then. Well, it happens that we have VRvana here in Montreal, a company that is developing their own VR/AR headset. Logically, we thought of customizing their headset, the Totem, in order to achieve what MindLeap is supposed to be doing. For the hackathon, our plan was to create a guided meditation VR/AR application. And that's what we did! On our table, we had a DIY respiratory belt, a pulse sensor, an OpenBCI board, EEG/ECG sensors, and, of course, the Totem. This headset seems designed for a VR/EEG dedicated set-up, specially because you can attach/detach velcro orange pads on the headset really easily (see pictures below). For the purpose of the hackathon we actually sewed a strap with the EEG sensors to one pad. In a near future, we can imagine different pads would be available on the market, each ones equipped with different sensors. By example, if I'm playing a game that doesn't require any sensors, I would mount the headset with the regular pad. If I want to play a guided meditation game that works with EEG or ECG, then I would replace the regular pad by the EEG/ECG pad, the hook and loop fastener system making it user friendly. We worked with OpenBCI and LabStreamingLayer for networking, time-synchronization, real-time access, and we built our scene in Unity3D.
Within a day, our team created a guided meditation game that works with respiratory belt, pulse, EEG & ECG. As the whole operation happened in a day, the actual game is pretty basic: the user starts in the augmented reality mode, a voice guides him and asks him to close his eyes, and open again. When the user opens his eyes, he is in the virtual reality mode, in an outdoor scene with a campfire in front of him. The user can't move, only look around. When he inhales, the campfire's intensity goes down, when the user exhales the campfire's intensity goes up. At the end of the session of meditation (3 minutes for the demo) the voice asks the user to close his eyes and open again, back in the augmented reality mode. All the data recorded from EEG/ECG, pulse and the belt during the sessions are archived and available through an application on the user's smartphone. Also, at the end of each session, the user can see an interpretation of all the datas recorded as a computer-generated painting (something between J. Pollock and fractals).
There is one main problem dealing with EEG, it is noise. The shorter will be your cables, the better. Rapid head movements can also generate noise, which is the last thing we want to record -even tho it is still a potentially useful input-. Otherwise noise, everything works well. You are more than welcome to download the project (soon online) and make it yours, please do! The more creators we are, the merrier, and the sooner we will be able to scientifically approve VR therapeutical potentialities.
Epilogue: That hackathon has been a blast! Our team is still working on the project. We can't wait to share the open project with the Github's community, probably within two weeks, once we adapted it for the Oculus Rift Dk2 -as for now the Totem headset is not available-. We actually won the first prize from the kids jury! The professional jury had a chance to try our demo on sunday morning and each ones definitively got the potential of neurogaming at large. I suspect we didn't get any prize from them because we didn't use the products that they were promoting for the hackathon: IBM's Watson and Bluemix. I loved it when my mate Jeremy told the organizers that we would certainly not use IBM engines as they are too slow for our project: "It's not good enough for our project, we deal with real time signals and your internet is too slow." Boom, in your face.
The team that won the first prize develops an application that lets parents aware of what is going on on their kids facebook' page. No needs to tell you what I'm thinking about that. Still, hackathons are a lot of fun and if you have a chance, just go for it! You would meet great and talented people that could help you greatly with your projects or, even better, become friends!