Category: Creative VR Project Portfolio

Now that i have managed to get most of the first difficulty working and in the Unity project fully playable I felt happy to start setting up the other difficulties for the project. The easy difficulty acted as a great framework where i could replicate what I had already created and scripted in unity but in a great scale for the other sections

Medium Difficulty

For the medium difficulty based on my script the user will need to create a four word sentence and so there are now four placeholders on the table in front of the player. For some added difficulty four new words have also been added to float around the player and the speed has also moderately been increased to add a slight sense of speed to the situation to make it more difficulty for the player to spot and grab the words quickly. This of course is how i want the mind to represent some of the difficulty if they are put into situations they find difficult to speak with.

Hard Difficulty

The hard difficulty is quite a sudden jump compared to the easy and medium difficulty with the user now having to create a six word sentence. There are now twenty four different words floating around the user and spinning at a pretty fast rate that will be quite difficulty for the user to find and grab the words within the time that the audio is becoming far more intense.

Tutorial (Starting Area)

One thing that I also need is the tutorial area that the player can stand in for as long as they want before starting the full experience. I created this with only the word ‘Hello’ floating around the player so this was very simple to create based on what I had made previously for the easy difficulty.

How to End the experience

This concept for a virtual reality experience could technically go on for a long time if you really wanted it to to have a massive mind full of different words and large sentence and typographic structures to create in a given time. However this experience needs to be short and so i feel that this could be ended in a way that I have personally not been able to finish conversations due to the anxiety and social mobility of certain conversations and situations.

It was suggested to me that the last word that the user needs to place could just never snap in place and it is impossible for the user to finish the sentence. This is a good representation of how difficulty it can sometimes be for us to speak and how we put ourselves in a corner and cant finish our sentences. This can carry on for a short while and eventually fade out over time leaving the player in a dark space showing that the player has given up trying to speak. A short piece of text can then be shown at the end of the experience explaining the situation and perhaps a way of showing awareness and a way of understanding how some people struggle with speaking.

“Stuttering is a speech disorder in which the flow of speech is disrupted by involuntary repetitions and prolongations of sounds, syllables, words, or phrases as well as involuntary silent pauses or blocks in which the person who stutters is unable to produce sound. Almost 70 million people worldwide stutter , under 1% of the worlds population with some receiving therapy but there is no cure for the disorder at present”

“Patience and understanding without judgement can show a small act of kindness in someones day”

The Ethics of this experience

I felt that this experience might have some ethical challenges due to the somewhat sensitive subject matter of what i am conveying to the user. While i personally have a stutter, i know that people deal with speech impediments in very different ways and some of course have different variations and intensities of speech disorders. So i want to add a disclaimer at the start of the experience to explain to the user what this is and how it is a abstract work of fiction:

Disclaimer – “The following is a work of fiction and based on the personal interpretation of one individual and should not be taken as a definitive representation of the human mind or a speech impediment such as a Stutter. Each person can experience a Stutter in many different ways and different levels of severity in different situations and stages of life.”

Creating The Flowers

For this idea I want the flowers to bloom out of the Vase like they are growing out of the imagery or the blank canvas on display. I have always wanted to try to create some 3D modelling in Blender or Maya so i decided that this would be a great way to follow some tutorials online to see what I can create.

I found multiple tutorials but one i felt was exactly what I needed where in the images below you can see the steps i followed to create a singular flower which had multiple petals that i thought would work well for this piece.

Blooming Animation

After following steps to create a flower model I also wanted to create some animations for the piece and using this tutorial it showed me how you could
achieve this. Using a cyclinder you can have the flower essentially squasged into a tall short petal to give the flower a blooming effect and by adding some of the physics options you can make this into quite a beautiful flowing effect which will
add to the vase greatly.

Modelling the flower was very interesting but I did struggle with some of the animation effects and exporting it as a useable file for unity. I do feel that i am confident in what I have learned this week in blender but i will need to develop this further for further unity tests and development to make this project successful.

After more developments and finalising my ideas for how i want the users to experience this ‘Mind of a stutterer’ I wanted to finish everything for the first level before i continue to develop the other difficulties. So i spent this time trying to polish and refine this so that it works in a playable state then hopefully this can be replicated on a larger scale when i try to make the other difficulties within the experience.

This required a series of trial and error to see if all the words would snap in place correctly for just 3 word options. Of course this will need to be increased with each subsequent difficulty or ‘level’

Initial Idea: Augmented Reality Vase

I have previously worked with a ceramics artist called Yuling when I visited her workshop in Tangshan, China back in 2019 and she is a fantastic sculptur and artist whos works I still have displayed. She was also very interested in ways that she could modernise her exhibitions using digital technology and I always thought it would be great to use AR as a way to not necessarily enhance her work but to accompany as an additional piece.

I felt that this would be a great idea to revisit and see if i could create an AR experience that could work with ceramic works. The idea for this piece would be using a vase or an image target on the vase which would then unlock an additional digital art piece that would be layered on top. So you could have a plain vase on display but when you scan it with your phone, you can see some animations of flowers or the imagery that is alread on display with the piece.

Adding Interactivity?

Of course AR experience can be triggered simple by observing an object with your movile device an the experience will begin but what if this could be enhanced further so that the user can actually ‘discover’ or ‘choose’ which animations on the vase would trigger first and when.

Downloading Vuforia

After coming up with some initial ideas for my Augmented Reality experience i needed to work out the basics of using Vuforia. So using the guide that was given to me i installed and imported the package to Unity which I had some problems at the start with different versions of the software not working but I eventually got this all working. As you can see from the images below I used some basic tests by using an image from my work notebook as a ‘image target’ and then used a simple model I have from turbosquid of a wallet as the augmented object.

From the images below after a few calibrations and some settings to change i was eventually able to get the AR experience working through my webcam so that when i held up my notepad to the camera the wallet model would appear on top. I am happy that this is now all working which means that i can progress with the next steps of trying to create my own AR animations/models to work for this experience.

During the development for this project I have had to rethink a few areas and how i can fully develop a polished experience within the given time frame and I have decided to shorten the experience to only having three levels of difficulty:

Level 01: Easy Difficulty – 3 Words
Level 02: Medium Difficulty – 4 Words
Level 03: Hard Difficulty – 6 Words

I think this will be an achieveable goal to create for this experience and all of these can still be created in one Scene or level just by having the different difficulties load in and out over given times within the game.

Stutter – Environment Script

For this experience i needed to create a simple conversation that could be discussed over three sentences while also conveying different levels of intensities within given conversations and also the frequency of words that will be floating around the player at any given time.

Level 01: Easy Difficulty

A: Hi, Really nice to see you again, how have you been?
P: Very Good Thankyou

This is a very basic level of difficulty where you will only need to create a sentence of three words and the conversation is simpler and approachable for a stutter.

Level 02: Medium Difficulty

A: Glad to hear, How has work been for you lately?
P: Stressful, but pushing through

This is a medium level of difficulty because while the conversation sounds simple its becoming more personal and perhaps the person wasnt ready for a question like this

Level 03: Hard Difficulty

A:oh okay, oh how is your partner lately, still together i trust?
P: Well Unfortunately, We broke up because…

This is where a conversation can become part of a hard difficulty and could become very difficult for a stutterer because not only is it a deeply personal question but could also throw someone off guard and perhaps not knowing a way out of the situation in a ‘socially acceptable’ way.

Game Design Document

Audio Recording

After writing the script for the experience I needed to have recordings of the dialogue to add to the Unity build. I do not personally know anyone else other than some family members who have a stutter and so I am going to have to record these lines because i want the experience to be as authentic as possible. My friend Alex very politely helped me by recording the lines of the person the stutterer is having a conversation with. All of the audio was then edited to be individual for the unity project and i wanted the audio for the conversation to be echoey to show it was coming from outside the mind which i was able to achieve through after effects.

Latest Environment Iteration

Looking through different models on turbosquid I came accross this great model of a cherry blossom tree which i felt suited atmosphere of the environment very well and i was able to add a new material to the object with ease. The environent needed to be typographic so I was able to create a material using images of book pages that would wrap around the tree model with very successful results, especially when it is shown in the dark environment. This material was then also used for the plan or surface that the user is standing on to again simulate being on a book page or part of a story or narrative.

The tree model was the duplicated and placed all around the environment to simulate the user being in a forrest inside a brancing mind which i feel also works very well especially when the whole experience is viewed through VR glasses.

Correct and Incorrect Word Interactivity

Last week i looked into the script required to create a correct word snapping effect for the different sentence structures for the whole experience. The current scene of the project is now set up to have eight different words currently floating around the player in a rather slow manner for the ‘easy difficulty’ section of the experience.

After further discussions with my peers and tutors it was suggested that i could also include a more obvious way of showing when the user has selected an incorrect word. A simple way of showing this is having the incorrect word highlight in red when it collides with the sentence box to show to the player that it is incorrect.

The script is used so that if a object tagged with ‘WrongWord’ enters the collider for one of the correct answer colliders, it will change its colour to red and on exit it will revert back to its original colours which I felt worked very well for the experience.

I initially wanted the words to snap back in place or for the animation to reset if the words were let go but it was suggested that perhaps it would add to the chaotic nature or environment if i turned off the gravity and the words would just float off which is something i will further experiment with in the future.

After some other peer assessment feedback, it was suggested that it would also be great if every time a user grabs a word, they would hear that word as well. So i started to attach some audio files of me saying the words which are floating around which was very useful feedback and something that I will carry forward with the development of this experience.

Over the passed week I began to look more into writing the scripts in unity for this project and what exactly i would need to achieve this type of experience. So to start working on this i decided to strip back the environment to the basics of justt having the words around the player and the objects in which they will snap to.

Levels of Difficulty

To create much more of a game element to this experience and to add more of a ‘fun’ factor rather than having the whole experience be just about how overwhelming a stutter can be on a persons mind I decided to think of the concept of different difficulty levels within the experience:

Level 01: Novice Difficulty – Create a 3 word setence
Level 02: Easy Difficulty – Create a 4 word setence
Level 03: Medium Difficulty – Create a 6 word setence
Level 04: Hard Difficulty – Create a 8 word setence
Level 05: Very Hard Difficulty – Create a 10 word setence

I felt that this would not only get the user use to the experience while not throwing them straight away into the deep end but this could also be a way to show different levels of stuttering in different types of conversations. This will create a much more ranged and interesting environment while also given the player something to aim for and do over the course of the experience.

Movement Animations

One of the first steps was to have the words actually float around the player in the environment which we were taught the basics of animation in Unity during this weeks classes. To test this out I decided to have multiple words to mess around with in the environment by using the starting word I made in Illustrator ‘Thankyou’.

Each word was parented with an empty in the centre of the environment that i named ‘Pivot’. This empty object contained an animation which would loop the word around the player in 360 degrees to have a continuous motion of moving around the player and the environment. This animation was then added to the other words and I decided to change around the movement and the starting position for all of these words and its interesting to see how this can be quite overwhelming or confusing in a virtual reality environment. After testing all of the different speeds i decided on how fast and the frequency of words that I wanted for the start of the experience to not overwhelm the player straight away and get them used to this environment.

Starting to Script in Unity

The next step after creating the initial animations was to try and find a way for the words to snap in place when the correct word is chosen and placed. I initially wanted the words that the player grabbed to snap in place on the ‘correct answer box’ when the object entered the collider but i struggled to find a way of doing this simply. I was then shown a different way of acheiving this by having a the correct answer hidden infront of the box and when the correct word is grabbed and placed, that word will dissapear and the hidden word would appear which would ‘simulate’ a snap in place function.

This was much harder to do than i originally thought however because I had some problems with the XR interaction tool hands. For a long time I was able to achieve the effect of moving the word into the trigger collider and having the hidden word appear but the word that was in hand would not dissapear! This was an error due to the object being held in the players hand so even when the console would say that the object was destroyed it would make a copy in the players hand for some reason. However over time we were able to resolve this issue so that the object would destroy and dissapear when then correct word was placed into that area.

I was very happy to eventually get this function working and each correct answer and correct word will need theyre own scripts to achieve this. However the next step is going to be finding a way to have the words that are floating around which arent correct to either snap back in place if the user lets go or have the animation reset or have a ‘fail’ option in some way.

Environment Setting and Assets

After achieving one of the base requirements for my experience I wanted to start thinking about how the environment of a ‘stutterers’ mind could look. Like i had previously said I wanted the environment to be very typographic in nature while it is also taking place in the mind and so my first thoughts is a world made of words and book pages.

Trees i think would also work well in this environment especially if they were made from words or book pages because this can represent a branching mind showing different paths and ways you can manipulate and control words in the mind. This is my own interpretation anyway of the mind of a stutterer which i feel will make a visually interesting experience for the user.

I started to look into multiple assets in the unity store and turbosquid for trees and forrest environments to try and simulate this effect which i need to continue researching to find different assets and options to create the environment i have envisioned in my head (literally).

Environment Atmosphere and Lighting

There were two options I had for the atmosphere of this environment to either keep it very ‘sepia tone’ and have the book pages fully on show with black text and white environments. However the other option would be to keep the environment very black and dark to have the words highlights and strongly on show. This would also help i think with the effect of being inside a dark mind and not being able to see the end of the void.

After trying different lighting options including having the players under a spot light and having the words shown over a dark background I think this is definately the best way to go for this experience.

After creating my initial idea for this VR environment and discussing the idea with my Tutors i was really excited to begin developing the concept in Unity. I also felt that for now calling the project ‘Stutter’ will be a good starting point based on the concept for this VR environment piece.

The Gameplay

Over the passed week i started to think about how i could create this experience and i decided to keep the interaction fairly straightforward so that the words will float around the user and when they grab the word they can place the word onto a given area to then snap in place and create a full sentence over time.

Importance of Audio

Stuttering is of course a very audible and sound based impediment and so I think that this is going to need to be a major part of this experience. Personally i find that when i want to say a certain sentence and i find myself unable to while stuttering, I can hear myself in my head trying so hard to say the sentence because you know what you want to say but your mouth simply cant get it out. I think that this was also shown brilliantly in the 2015 short film ‘Stutterer’ by Benjamin Cleary in which the main character is shown struggling with speaking but you can hear his internal monologue explaining what he wants to say and how strongly he wants to.

I think that this was done very well and this is something that I would like to do with the audio for my Virtual reality experience. This means that when the user is playing perhaps over time they will hear an internal monologue of the sentence that they need to create but the longer they take (similar to talking in real life) the more the sentences in the user head will overlap and the pressure will be built up to make the sentence in a ‘socially acceptable’ amount of time.

Creating the Environment

I know that for this environment I wanted to keep the entire experience to be very typographic and of course based on reading, hearing and language and so i started to think about how the user will put words together infront of them. After looking through the Unity asset store I found a model for a table which reminded me personally of a typographers table that i felt would suit this experience very well.

Adding Interactivity

I started by downloading and importing the Unity XR interaction tool which is extremely useful due to the preinstalled assets and features it has included for Virtual Reality projects. I then imported the typographers table and placed a few blocks to represent the area that the player will try to place and create words.

To start testing the concept using Illustrator I created the word ‘Thankyou’ as a PNG that could added into the build as a Sprite. Each word has a box collider which the user can then grab and move around to try and create sentences. The idea for this environment will be for the user to grab one of the floating words and place it accordingly on the correct sentence block so if the sentence is only three words, then they will need to create that sentence on the table infront of them over that given them etc.

What to do next?

I feel that I have this project concept and idea in a good place to continue the development but i will need to look to start writing Unity scripts to fully achieve this idea and concept which I will research into and of course continue the weekly Unity classes in the meantime.