1 Research1.1 VR for psychology1.2 Food & VR?1.3 I am missing the beach 🏝 1.4 What I’m going to miss back in The Hague 🇳🇱2 Tinkering2.1 System restrictions with Roblox2.2 Roblox Any game with Google Cardboard2.3 Spinning kebab in A-Frame2.4 Tinkering in Roblox3 Design & Build3.1 The tridirectional quarantine framework3.2 Continuing in Roblox3.3 Putting up the kebab skew3.4 A good spin is key (and so is a good weld anchoring)3.5 Implanting sound objects for a spheric 3D sound experience3.6 Installing Windows 3.7 The video: Showcasing the instant satisfaction jaccuzi3.8 Immerse yourself4 Reflection
1 Research
During my research phase I conducted in the inner friends & family circle I found there are certain things me and my friends are missing in the current crisis.
1.1 VR for psychology
In the guest lecture with Mr. Maureira he adressed the huge potential for VR in terms for applications with psychological impact (e.g. treatments for people with mental disorders) - but otherwise he sees more potential in AR technology.
But aren't we all suffering from the crisis already? What might the long-term impact look like?
I am aiming to create an immersive experience for all those who suffer from the current situation.
This week, I've spent two days in hospital 🏥. Before my treatment I was bedded in a waiting room 🛌. The patient next to me was connected to a oxygen pump, which to me just made the exact same sound as a jacuzzi pump distributing bubbles in a hot whirlpool. Besides that, they played holiday charts music from the 2000s. When I closed my eyes, I suddenly felt incredibly immersed by those two ingredients and immediatly flashed back to my last holiday in Spain 🇪🇸. My treatment was done under full anesthesia - before they induced the medication the doctors engaged me into a conversation. My last words were "wow, am I getting tired right now?" - "Yes, the doctor answered" I woke up 4 hours later in the initial room, with the same relaxing music, with the oxygen jacuzzi pump still running. I was entirely confused and my initial thoughts were "did someone just respawn me - is this real? Hey doctor, what time is it" - "2.30pm - You have been asleep for four hours" This entire journey showed me, how simple our imagination can be supported by stimulating senses either by speech, sounds, music or narcotic drugs.
I am curious, how adding the visual sense might either improve or worsen this experience, as recalling events from our visual memories is often more personal, meaningful and prioritizes positive fragments. However I assume that inserting random visuals could even destroy the immersive experience my brain would generate when entirely relying on sound stimulation.
1.2 Food & VR?
But what is meaningful during the crisis? As restaurants were closed for a while, I was craving nothing more but a Döner Kebab. I know - you might say "Do it yourself at home fgs" - True, that's what I am already doing. But it is not the same. Going to your local Döner Kebab shop means, yet again, immersion: You enter the shop, by saying "Salam aleykum", the kebab man answers "Guten Tag", in the background there is high-pitched oriental music playing. The raw kitchen tools and fast working pace just contributes to an overall satisfying atmosphere. And I am missing this atmosthere.
The Kebab Simulator VR video was especially interesting to see, as it captured the experience just from the outside, while the VR experience itself was not shown in the video and thus pretty much up to the viewers interpretation. I see this as interesting entry point for my tinkering phase.
1.3 I am missing the beach 🏝
Yet another aspect that I am missing in the current phase is the Scheveningen beach - especially the ride towards there. That comes together with the bad bicycle infrastructure in Germany. Whenever I go for a bike ride here, it requires extensive planning. A bicycle helmet is required in any case as some people, especially from the countryside who never saw a bicycle in their life tend to behave like maniacs. Planning the route upfront is therefore crucial. Dealing with the given issues, you might understand how higher my demand for relaxation here in Germany is .
1.4 What I’m going to miss back in The Hague 🇳🇱
All the cat cafes in the randstad area closed for some obscure reason. I am going to miss a good purring cat.
2 Tinkering
Initially I planned on using Roblox for the prototype. The reason for choosing Roblox was primiraly due to the fact, that in the parallel course "PS1 - Shared Spaces", in my team, I was the responsible person for the 3D prototype and while the HCI course started, as it was the last big chunk in our project, I still had enough work load allocated there.
Gaining advanced knowledge both in Unreal and Roblox at the same time would have been great, however was just infeasible due to the limited amount of time.
2.1 System restrictions with Roblox
After finally having free capacities for this project I realized the only VR headsets natively supported by Roblox apparently are the more elaborate ones (HTC Vive, Oculus Rift). However as for the peer review it would be desirable being able to test the entire experience, this aspect meant a huge setback for me.
2.2 Roblox Any game with Google Cardboard
As I didn't want to give up just yet, I conducted some more research on how to get Roblox or any other game running in "budget"-VR Mode.
Quite soon I mapped out two different tools that seemed to allow this ability
- Riftcat https://riftcat.com/vridge
- Trinus VR https://www.trinusvirtualreality.com/
The euphoria however was not there to stay as I found that all the reserached tools, although compatible with iOS, were Windows only - neither there is MacOS, nor Linux support.
But should I go now throw the towel just due to some companies dictating me into special ecosystems or platforms? Should I replan everything and move to AR? Which AR engine? Apple ARKit which is limited to Apple Devices? Google ARCore, which is limited to Android devices?
No, I said - this won’t bring me down from my path. I want to build something that experience-wise works “as cross-platform as possible.”
2.3 Spinning kebab in A-Frame
I started my tinkering session in A-Frame as I like their minimalistic approach to be fully cross-platform and available on the minimal possible hardware. I see this as success factor on the road towards mass adoption of the kebab simulator ;)
However as I wanted to achieve a clear focus here in immersing the user both with sound and visuals, there was an unforeseen obstacle. Sound support in the iOS mobile browser (a good thing in general, interaction of the user gives consent for potentially unwanted sound) The Support doc mentioned, that this can be solved by including a button. Fingers crossed, how it works out - see you in my kebab shop.
First came the tiles - as in every kebab shop the standard gastronomy tiles can't be missing
2.4 Tinkering in Roblox
I found that in Roblox you can easily add and adjust objects from the community - but of course your own OBJ / FBX files as well. Adjusting is possible in two ways:
- Adjusting the mesh / object structure & material
- Make it interactive by using C++ code
Adding code to 3D objects seemed quite powerful - I saw driving cars, flying seegulls, even flyable drones. That gave great inspiration on what, interaction-wise is possible here.
3 Design & Build
Before building the concept itself, I wanted to make sure I know what to build. In order to get an overview I created a brief concept / framework, where you can clearly see my priorities as well. Sound stimulation clearly goes over visual stimulation, as I first want to see, whether the sound immersion works and then make the visual part fit. This is particularly interesting as I want my solution to work as well when closing my eyes.
3.1 The tridirectional quarantine framework
A tridirectional... what? No, in the following section I won’t describe yet another ponzi-scheme I want you to sign up for - it is rather creativity caused by technical constraints. But let me explain...
When working with "budget"-VR googles like the Google Cardboard, we are pretty much constrained to a fixed location, as there are no controls besides e.g. from "hovering" objects. I therefore decided to split up my experience in different directions
- Kebab corner ( A spinning kebab skew emitting kebab pieces to my plate, while playing oriental music )
- Beach corner ( Relaxing wave sounds)
- Cat corner (Purring cat sounds)
3.2 Continuing in Roblox
What I was still missing for the "tridirectional" experience was a type of foundation. But what is typical for a beach? A towel? Maybe. However I decided to go for a more posh approach, as I was still missing the link to the jacuzzi sounds mentioned above.
3.3 Putting up the kebab skew
That's going to be easy, right? I placed a banana on a platform before in Unreal - so I did follow a similar way of modeling here. A skew and a cylinder shape with texture. But what happened there?
That skew became victim of gravity. I had to do some research on how to attach objects to another object.
The key to success was brought me by the welding function.
And - tadá, the kebab skew was attached to the pool. But was it spinning?
3.4 A good spin is key (and so is a good weld anchoring)
The kebab skew was attached - but was it spinning? Not yet.
while true do wait() local BRICK = script.Parent BRICK.CFrame = BRICK.CFrame * CFrame.fromEulerAnglesXYZ(0.09,0,0) end
Adding a few lines of code that basically tell the "Parent" object, the object where the script is placed in, to start spinning solved it. Well, to an extent.
But somehow, the pool did the same thing. So I kept on trying to attach the skew to other objects. Either it fell down again or the parent object started spinning as well. And actually that seems to follow a clear logic. So I dug deeper and found: Aha, anchoring is the way of doing it.
The kebab suddenly started spinning for itself - a great moment.
3.5 Implanting sound objects for a spheric 3D sound experience
3D sound is tricky - I never really dealt with it before. But luckily Roblox has way to implant sound into objects:
After following the tutorial above, I managed to get the "3D" sound working. By defining different distances and sound levels, there is now a perceivable difference regarding sound level when moving the player's head around.
3.6 Installing Windows
In order to get Roblox running with Budget-VR goggles, I needed to install the Trinus VR Client mentioned above and thus fired up a Windows virtual machine.
This however brought up the next challenge: Network problem with ports in virtual machines. Probably I would need to find out how to bridge the required Port 7000 from my virtual machine to the virtual macOS adapter so that my phone would be able to find the server. After 1h of troubleshooting I postponed that and carried on with the prototype.
3.7 The video: Showcasing the instant satisfaction jaccuzi
The video explains the overall objective behind the technology I was building during the last two weeks and captures a user being immersed with the interactive VR environment.
Prototype by Phil Wornath, created in the scope of the course "Human Technology" at The Hague University.
3.8 Immerse yourself
Why watching videos, when you could experience the immersion yourself 😉Come and drop by at my Roblox place and have a jacuzzi session with cats and kebabs on me.
4 Reflection
Working with the Unreal Engine has been interesting, however it was a very bloating software taking almost 60GB of storage from my running machine. This would have been handy to know upfront so I could have taken measures before - or complete my other university projects before installing this heavy piece of software. But now it interfered with other projects (video, other 3d game development) running alongside, so after all I had to make the decision to remove it
This was particularly hard for me as I have a considerable better setup in The Netherlands but due to Corona I was limited to my minimalistic home office setup which I absolutely did not prepare for the assignments we suddenly had to do. I constantly needed to clean storage and it interrupted my flow, slowed down the switching between my projects and overall discouraged me to think outside the boundaries.
Besides other courses, that had the be completed in parallel (which is rather a planning issue), my stay in hospital last week but also the heavyiness of these tools were the major reasons, why I kept on moving the creation of my VR prototype and ended up creating it quite short before the peer review which was rescheduled to earlier than announced earlier.which was rescheduled to earlier than announced earlier.
Next time, I would highly appreciate knowing upfront we are going to work with VR in general and go beyond web-based frameworks like A Frame as well. I postponed the Unreal assignment for quite a while but then realized after all we were allowed to use different software as well for our challenge.
I am here in Home office mode with minimal setup and 1st gen. iPhone SE which is not really great to do testing with,sue to the low resolutionfor my.
It's not me hatespeeching against VR/AR - I always try to be enthusiastic and open towards technologies but besides the fact that my interest in IoT / Arduino is way higher, while the required setup in comparison can be very minimal, In my honest opinion: Starting with AR/VR is a bit overkill for the first weeks, especially as we had to finish other courses before.
Overall I enjoyed meeting new technologies but it seemed way to rushed with other courses in between. Estimating half of the work effort for courses that are in the wrapping up-phase (where most of the workload happens, at least in group work) is probably too optimistic. I hope you can consider this.
Besides that I was happy with the outcome, although I will definitely experiment further with AR in particular. Maybe there are some nice synergies to use AR applied to Smart Home to dive deeper .