Stutter – Developing Medium and Hard Difficulty Levels

Now that i have managed to get most of the first difficulty working and in the Unity project fully playable I felt happy to start setting up the other difficulties for the project. The easy difficulty acted as a great framework where i could replicate what I had already created and scripted in unity but in a great scale for the other sections

Medium Difficulty

For the medium difficulty based on my script the user will need to create a four word sentence and so there are now four placeholders on the table in front of the player. For some added difficulty four new words have also been added to float around the player and the speed has also moderately been increased to add a slight sense of speed to the situation to make it more difficulty for the player to spot and grab the words quickly. This of course is how i want the mind to represent some of the difficulty if they are put into situations they find difficult to speak with.

Hard Difficulty

The hard difficulty is quite a sudden jump compared to the easy and medium difficulty with the user now having to create a six word sentence. There are now twenty four different words floating around the user and spinning at a pretty fast rate that will be quite difficulty for the user to find and grab the words within the time that the audio is becoming far more intense.

Tutorial (Starting Area)

One thing that I also need is the tutorial area that the player can stand in for as long as they want before starting the full experience. I created this with only the word ‘Hello’ floating around the player so this was very simple to create based on what I had made previously for the easy difficulty.

How to End the experience

This concept for a virtual reality experience could technically go on for a long time if you really wanted it to to have a massive mind full of different words and large sentence and typographic structures to create in a given time. However this experience needs to be short and so i feel that this could be ended in a way that I have personally not been able to finish conversations due to the anxiety and social mobility of certain conversations and situations.

It was suggested to me that the last word that the user needs to place could just never snap in place and it is impossible for the user to finish the sentence. This is a good representation of how difficulty it can sometimes be for us to speak and how we put ourselves in a corner and cant finish our sentences. This can carry on for a short while and eventually fade out over time leaving the player in a dark space showing that the player has given up trying to speak. A short piece of text can then be shown at the end of the experience explaining the situation and perhaps a way of showing awareness and a way of understanding how some people struggle with speaking.

“Stuttering is a speech disorder in which the flow of speech is disrupted by involuntary repetitions and prolongations of sounds, syllables, words, or phrases as well as involuntary silent pauses or blocks in which the person who stutters is unable to produce sound. Almost 70 million people worldwide stutter , under 1% of the worlds population with some receiving therapy but there is no cure for the disorder at present”

“Patience and understanding without judgement can show a small act of kindness in someones day”

The Ethics of this experience

I felt that this experience might have some ethical challenges due to the somewhat sensitive subject matter of what i am conveying to the user. While i personally have a stutter, i know that people deal with speech impediments in very different ways and some of course have different variations and intensities of speech disorders. So i want to add a disclaimer at the start of the experience to explain to the user what this is and how it is a abstract work of fiction:

Disclaimer – “The following is a work of fiction and based on the personal interpretation of one individual and should not be taken as a definitive representation of the human mind or a speech impediment such as a Stutter. Each person can experience a Stutter in many different ways and different levels of severity in different situations and stages of life.”

AR Ceramics – Initial unity development and animations

Creating The Flowers

For this idea I want the flowers to bloom out of the Vase like they are growing out of the imagery or the blank canvas on display. I have always wanted to try to create some 3D modelling in Blender or Maya so i decided that this would be a great way to follow some tutorials online to see what I can create.

I found multiple tutorials but one i felt was exactly what I needed where in the images below you can see the steps i followed to create a singular flower which had multiple petals that i thought would work well for this piece.

Blooming Animation

After following steps to create a flower model I also wanted to create some animations for the piece and using this tutorial it showed me how you could
achieve this. Using a cyclinder you can have the flower essentially squasged into a tall short petal to give the flower a blooming effect and by adding some of the physics options you can make this into quite a beautiful flowing effect which will
add to the vase greatly.

Modelling the flower was very interesting but I did struggle with some of the animation effects and exporting it as a useable file for unity. I do feel that i am confident in what I have learned this week in blender but i will need to develop this further for further unity tests and development to make this project successful.

Mind the Gap – Storyboard Developments (Tiltbrush)

After practicing with tilt-brush i wanted to delve deeper with creating my storyboard and actually filling the full train carriage with more of the puzzles, prompts and direction that the player will follow in the experience. Over the passed week I was able to draw the storyboard for the main three puzzles of the experience with the starting and ending area I hope to work on later.

Puzzle 02 – Test of Wits

At the start of the carriage i drew out the second user task in which the player will need to press four buttons around this area of the carriage in the correct sequence. I have indicated this by drawing the four buttons and using the highlight brush, i have circled each one to show its location and also drawing the number order in which they need to be pressed.

Puzzle 03 – Test of Strength

In the middle section of the carriage I have drawn out the task for the test of strength by drawing multiple blocks that are block the way for the passenger. I have then drawn multiple arrows in different directions to show that each object needs to be moved in order for the player to progress further. This is also indicated by the text prompt I have written above as the player will enter this section of the experience.

Puzzle 04 – Test of Speed

At the end of the carriage I have drawn out the task for the Test of Speed in which the player will need to pull down multiple hand rails in a given time in order to progress to the end of the carriage. I have indicated this by drawing out the hand rails and then highlighting each one with a downwards arrow indicating that each one will need to be pulled down by the player. of course this was further highlighted by some text at the start of the experience briefly explaining what the player will need to do next.

Storyboard Next Steps

I feel more confident now in using tilt-brush and feel i know what i need to do further to accomplish my full storyboard. I will of course need to complete first train carriage showing it in both states containing random NPC characters and one without NPC’s with the objects scattered around the environment. I will also need to create or find an asset for the drivers carriage showing the ‘Antagonist’ and also showing the elements of dialogue and the ending for the experience. Luckily I should be able to import some assets and objects from my initial unity build.

Stutter – Finalising the first difficulty in Unity

After more developments and finalising my ideas for how i want the users to experience this ‘Mind of a stutterer’ I wanted to finish everything for the first level before i continue to develop the other difficulties. So i spent this time trying to polish and refine this so that it works in a playable state then hopefully this can be replicated on a larger scale when i try to make the other difficulties within the experience.

This required a series of trial and error to see if all the words would snap in place correctly for just 3 word options. Of course this will need to be increased with each subsequent difficulty or ‘level’

AR Experience – Initial Concept and Ideas / Vuforia Tests

Initial Idea: Augmented Reality Vase

I have previously worked with a ceramics artist called Yuling when I visited her workshop in Tangshan, China back in 2019 and she is a fantastic sculptur and artist whos works I still have displayed. She was also very interested in ways that she could modernise her exhibitions using digital technology and I always thought it would be great to use AR as a way to not necessarily enhance her work but to accompany as an additional piece.

I felt that this would be a great idea to revisit and see if i could create an AR experience that could work with ceramic works. The idea for this piece would be using a vase or an image target on the vase which would then unlock an additional digital art piece that would be layered on top. So you could have a plain vase on display but when you scan it with your phone, you can see some animations of flowers or the imagery that is alread on display with the piece.

Adding Interactivity?

Of course AR experience can be triggered simple by observing an object with your movile device an the experience will begin but what if this could be enhanced further so that the user can actually ‘discover’ or ‘choose’ which animations on the vase would trigger first and when.

Downloading Vuforia

After coming up with some initial ideas for my Augmented Reality experience i needed to work out the basics of using Vuforia. So using the guide that was given to me i installed and imported the package to Unity which I had some problems at the start with different versions of the software not working but I eventually got this all working. As you can see from the images below I used some basic tests by using an image from my work notebook as a ‘image target’ and then used a simple model I have from turbosquid of a wallet as the augmented object.

From the images below after a few calibrations and some settings to change i was eventually able to get the AR experience working through my webcam so that when i held up my notepad to the camera the wallet model would appear on top. I am happy that this is now all working which means that i can progress with the next steps of trying to create my own AR animations/models to work for this experience.

Mind the Gap – Script Second Draft & Story-boarding Attempt 2

After my first attempt with creating a virtual reality storyboard in tilt brush and quill i knew that i would need to practice and find better ways to visualise my script and experience within this format. So after doing some research I found out that you can import 3D models and objects into tilt brush which I could then use as either a template, point of reference or a backdrop/frame for drawing my storyboard.

Importing My Assets

So i began by importing the train model that i had previously found on tilt brush that i would like to use for my experience and used this as a framework for the hand drawn elements. I wanted to start simple by trying to create one scene or frame from the script which was the fourth puzzle in which the user will need to pull down the hand rails in order to progress to the drivers carriage to end the story.

For practice and my first developments I practiced using the tilt brush controls and started to learn what tools were available for me to use which was really fun to try. I kept it simple and drew the hand rails above the seats in the environment and using the highlight brush drew around these and also drew a downwards arrow to indicate to the player that they need to pull all of these down to progress. I then drew an arrow in the direction that the character needs to move to progress further.

Next Steps

This is still not quite where i want this to be but i feel better about using the tilt brush software now that i know how to import models and assets into my tilt brush file. I know how to progress further with this and will continue to try and draw the rest of my experience over the next week and have it in a more finalized state for review by adding more detail, text prompts and other elements that i feel could enhance this and convey my story.

SECOND DRAFT OF SCRIPT

During the passed week i tried to critique my first draft of the script and tried to think of ways to refine the narrative, improve, remove or any additions that i think it might have needed. After reading through the script multiple times I decided that maybe the character that the player is embodying should speak and should be presented as a silent protagonist. This would mean that the player could perhaps embody the experience even more if they are playing themselves during the whole experience.

This meant that I needed to further rethink the dialogue of the train announcer so that the announcer would talk and present the story to the player without the player needing to answer back. After making this change to the script i feel this works better as a more immersive experience and it also reduces the amount of dialogue required keeping the experience shorter as per the project brief.

Stutter – Audio and Further Unity Tests & Environment Development

During the development for this project I have had to rethink a few areas and how i can fully develop a polished experience within the given time frame and I have decided to shorten the experience to only having three levels of difficulty:

Level 01: Easy Difficulty – 3 Words
Level 02: Medium Difficulty – 4 Words
Level 03: Hard Difficulty – 6 Words

I think this will be an achieveable goal to create for this experience and all of these can still be created in one Scene or level just by having the different difficulties load in and out over given times within the game.

Stutter – Environment Script

For this experience i needed to create a simple conversation that could be discussed over three sentences while also conveying different levels of intensities within given conversations and also the frequency of words that will be floating around the player at any given time.

Level 01: Easy Difficulty

A: Hi, Really nice to see you again, how have you been?
P: Very Good Thankyou

This is a very basic level of difficulty where you will only need to create a sentence of three words and the conversation is simpler and approachable for a stutter.

Level 02: Medium Difficulty

A: Glad to hear, How has work been for you lately?
P: Stressful, but pushing through

This is a medium level of difficulty because while the conversation sounds simple its becoming more personal and perhaps the person wasnt ready for a question like this

Level 03: Hard Difficulty

A:oh okay, oh how is your partner lately, still together i trust?
P: Well Unfortunately, We broke up because…

This is where a conversation can become part of a hard difficulty and could become very difficult for a stutterer because not only is it a deeply personal question but could also throw someone off guard and perhaps not knowing a way out of the situation in a ‘socially acceptable’ way.

Game Design Document

Audio Recording

After writing the script for the experience I needed to have recordings of the dialogue to add to the Unity build. I do not personally know anyone else other than some family members who have a stutter and so I am going to have to record these lines because i want the experience to be as authentic as possible. My friend Alex very politely helped me by recording the lines of the person the stutterer is having a conversation with. All of the audio was then edited to be individual for the unity project and i wanted the audio for the conversation to be echoey to show it was coming from outside the mind which i was able to achieve through after effects.

Latest Environment Iteration

Looking through different models on turbosquid I came accross this great model of a cherry blossom tree which i felt suited atmosphere of the environment very well and i was able to add a new material to the object with ease. The environent needed to be typographic so I was able to create a material using images of book pages that would wrap around the tree model with very successful results, especially when it is shown in the dark environment. This material was then also used for the plan or surface that the user is standing on to again simulate being on a book page or part of a story or narrative.

The tree model was the duplicated and placed all around the environment to simulate the user being in a forrest inside a brancing mind which i feel also works very well especially when the whole experience is viewed through VR glasses.

Correct and Incorrect Word Interactivity

Last week i looked into the script required to create a correct word snapping effect for the different sentence structures for the whole experience. The current scene of the project is now set up to have eight different words currently floating around the player in a rather slow manner for the ‘easy difficulty’ section of the experience.

After further discussions with my peers and tutors it was suggested that i could also include a more obvious way of showing when the user has selected an incorrect word. A simple way of showing this is having the incorrect word highlight in red when it collides with the sentence box to show to the player that it is incorrect.

The script is used so that if a object tagged with ‘WrongWord’ enters the collider for one of the correct answer colliders, it will change its colour to red and on exit it will revert back to its original colours which I felt worked very well for the experience.

I initially wanted the words to snap back in place or for the animation to reset if the words were let go but it was suggested that perhaps it would add to the chaotic nature or environment if i turned off the gravity and the words would just float off which is something i will further experiment with in the future.

After some other peer assessment feedback, it was suggested that it would also be great if every time a user grabs a word, they would hear that word as well. So i started to attach some audio files of me saying the words which are floating around which was very useful feedback and something that I will carry forward with the development of this experience.

Mind the Gap – Script First Draft and Unity Assets/Environment

UNITY DEVELOPMENTS

This module is more focused on the script and storyboard but we also need to create a unity build environment of our experience. I have already found some train carriage options for my experience but I then started by looking on the Unity Asset Store and turbo squid to see what assets I could find which will be useful for my experience. I began by trying out some of the different train models that I had found on turbo squid and seeing how they look and felt in the Unity environment as shown in the screenshots below.

First Puzzle Assets

I knew that for my first puzzle I wanted the user to be tested on their personality by having different objects around them to choose from each representing a different type of person. For this idea i had chosen these options:

Option 01: Wallet – This object represents the player choosing wealth or greed

3d leather wallet https://static.turbosquid.com/Preview/2015/04/16__12_14_36/LeatherWallet3dmodel00.jpg17719be8-3a21-4535-a16d-8385c4c4335cOriginal.jpg

Option 2: Watch – This object represents the player choosing time

watch citizen 3d model https://static.turbosquid.com/Preview/000241/400/BD/watch-citizen-3d-model_D.jpg

Option 03: Book – This object represents the player choosing knowledge

https://static.turbosquid.com/Preview/000321/013/ZS/_0.jpg

Setting and Lighting

The entirety of this experience takes place on the London underground or a train subway and so the whole environment will always be in a dark train tube. To simulate the train moving through the underground I thought it would work very well if the whole sky box and environment was pitch black with only the lights inside the carriage on display and also a light that will pass by the train. The light can be on a repeated animation of continuously passing the trains windows on both sides to simulate the look of the train moving and passing the different lights in the tunnel.

SCRIPT DEVELOPMENTS – FIRST DRAFT

Over the passed week i have made further developments with my script where I feel I have a first draft ready for review. I discussed my current story progress with my tutors who gave me very useful feedback that perhaps I have made the story too subtle and perhaps the player could use some more guidance and more answers to this situation to elevate what they need to do and why they are playing this experience in Virtual reality rather than through a traditional gaming experience. It was also suggested to me that my puzzles perhaps needs more meaning, purpose and driving force for the whole experience. This feedback was extremely useful and really helped me to further develop my idea over the passed week.

Rethinking the narrative

One of the first elements i decided to change in my script was the narrative giver which is of course the train announcer. I initially planned for the train announcer to be kept in the traditional voice you would find on the London underground but I felt that there could be a much more fun way to convey the story. I decided to change the voice so that the actually spirit/entity is talking directly to the player and mysteriously takes over the voice to add mystery and tension for the player.

I wanted the spirit to be fun and in a way taunt the player to try and get them to be faster and rush through the experience. I wanted to make this character somewhat sarcastic and showing a sense of boredom urging the player to finish the tasks as quick as possible to prove themselves while also entertaining the spirit. In the script i have also decided to add a narrative for the player as well so they will embody a character which will talk back to the train announcer showing tension, un-ease and frustration with the person that is testing them trying to figure out what is happening. I hope that this will indicate and add a sense of urgency for the player during the duration of the experience.

Rethinking the puzzles

After receiving feedback on my idea I knew that i needed to add more meaning or narrative behind each of the puzzles and so for this reason i have based all of them around a structure while adding one more additional puzzle to the mix.

The whole experience is based around a spirit/entity testing the player for his own enjoyment and to see how much the player wants to live and so i thought it would be fun to add a ‘meaning’ or ‘purpose’ behind each one.

A Test of Hearts – This first test will remain the same in which the player will need to choose an object that represents their ‘heart’ except I have added an additional object so their will be a ‘Book’, ‘Watch’, ‘Wallet’ and ‘Book’ to have a wide range.

A Test of Wits – This will be the button sequence puzzle in which the players wits or intelligence will be tested and they will need to choose the correct order of strongest to weakest of four different chess pieces shown on buttons around the carriage.

A Test of Strength – This is a new puzzle in which the player will need to move multiple objects out the way of the entrance to the next section and this will act as a simple test of strength and willingness to progress to the next area.

A Test of Speed – This will be the pulling lever sequence in which the player will have a short amount of time to pull down all of the hand rails that are in the last section of the carriage to represent the players test of speed to get off the train.

I thought these all come together very well to show that the spirit is taunting and literally testing the player while they are completing each of the tasks which hopefully over time conveys the story of what is happening to player over the duration of the short experience.

Stutter – Interactivity and developing the environment

Over the passed week I began to look more into writing the scripts in unity for this project and what exactly i would need to achieve this type of experience. So to start working on this i decided to strip back the environment to the basics of justt having the words around the player and the objects in which they will snap to.

Levels of Difficulty

To create much more of a game element to this experience and to add more of a ‘fun’ factor rather than having the whole experience be just about how overwhelming a stutter can be on a persons mind I decided to think of the concept of different difficulty levels within the experience:

Level 01: Novice Difficulty – Create a 3 word setence
Level 02: Easy Difficulty – Create a 4 word setence
Level 03: Medium Difficulty – Create a 6 word setence
Level 04: Hard Difficulty – Create a 8 word setence
Level 05: Very Hard Difficulty – Create a 10 word setence

I felt that this would not only get the user use to the experience while not throwing them straight away into the deep end but this could also be a way to show different levels of stuttering in different types of conversations. This will create a much more ranged and interesting environment while also given the player something to aim for and do over the course of the experience.

Movement Animations

One of the first steps was to have the words actually float around the player in the environment which we were taught the basics of animation in Unity during this weeks classes. To test this out I decided to have multiple words to mess around with in the environment by using the starting word I made in Illustrator ‘Thankyou’.

Each word was parented with an empty in the centre of the environment that i named ‘Pivot’. This empty object contained an animation which would loop the word around the player in 360 degrees to have a continuous motion of moving around the player and the environment. This animation was then added to the other words and I decided to change around the movement and the starting position for all of these words and its interesting to see how this can be quite overwhelming or confusing in a virtual reality environment. After testing all of the different speeds i decided on how fast and the frequency of words that I wanted for the start of the experience to not overwhelm the player straight away and get them used to this environment.

Starting to Script in Unity

The next step after creating the initial animations was to try and find a way for the words to snap in place when the correct word is chosen and placed. I initially wanted the words that the player grabbed to snap in place on the ‘correct answer box’ when the object entered the collider but i struggled to find a way of doing this simply. I was then shown a different way of acheiving this by having a the correct answer hidden infront of the box and when the correct word is grabbed and placed, that word will dissapear and the hidden word would appear which would ‘simulate’ a snap in place function.

This was much harder to do than i originally thought however because I had some problems with the XR interaction tool hands. For a long time I was able to achieve the effect of moving the word into the trigger collider and having the hidden word appear but the word that was in hand would not dissapear! This was an error due to the object being held in the players hand so even when the console would say that the object was destroyed it would make a copy in the players hand for some reason. However over time we were able to resolve this issue so that the object would destroy and dissapear when then correct word was placed into that area.

I was very happy to eventually get this function working and each correct answer and correct word will need theyre own scripts to achieve this. However the next step is going to be finding a way to have the words that are floating around which arent correct to either snap back in place if the user lets go or have the animation reset or have a ‘fail’ option in some way.

Environment Setting and Assets

After achieving one of the base requirements for my experience I wanted to start thinking about how the environment of a ‘stutterers’ mind could look. Like i had previously said I wanted the environment to be very typographic in nature while it is also taking place in the mind and so my first thoughts is a world made of words and book pages.

Trees i think would also work well in this environment especially if they were made from words or book pages because this can represent a branching mind showing different paths and ways you can manipulate and control words in the mind. This is my own interpretation anyway of the mind of a stutterer which i feel will make a visually interesting experience for the user.

I started to look into multiple assets in the unity store and turbosquid for trees and forrest environments to try and simulate this effect which i need to continue researching to find different assets and options to create the environment i have envisioned in my head (literally).

Environment Atmosphere and Lighting

There were two options I had for the atmosphere of this environment to either keep it very ‘sepia tone’ and have the book pages fully on show with black text and white environments. However the other option would be to keep the environment very black and dark to have the words highlights and strongly on show. This would also help i think with the effect of being inside a dark mind and not being able to see the end of the void.

After trying different lighting options including having the players under a spot light and having the words shown over a dark background I think this is definately the best way to go for this experience.

Mind the Gap – Initial Script Development and VR Storyboarding

INITIAL SCRIPT WRITING

Now that I had my idea ready I was ready to start writing the story and script for the full experience. I started by looking at different options that I could look into for the puzzles/tasks in the experience. Of course i need to remember to keep this short experience simple and short and so I cant have the puzzles too complex or difficulty to understand so i started with these puzzle options:

Puzzle 01 – Grab and Place object

For the first puzzle the user will need to pick between three different objects that represent their personality depending on their choice. When they have grabbed one of these objects they will need to place the object onto a panel next to the door to the next carriage which will confirm their choice while also opening the door to the next section.

Puzzle 02 – Button Sequence

The button sequence puzzle will involve the player having to press different buttons around the carriage in a certain sequence in order to open the next section of the carriage to progress. My idea for this is again based on the users personality of having to choose in order which seasons of the year they prefer from their favorite to their least favorite. (Choice between Spring, Summer, Autumn and Winter)

Puzzle 03 – Pull Levers

The final puzzle will be for the player to pull down different hand rails or levers around the last section of the carriage in a certain sequence. The answer to this sequence could be shown as a poster or a visual queue somewhere around the carriage so the player will need to be observant of their surroundings.

Train announcer

I want the train announcer to be the main driving force for the full narrative and so I felt it would be interesting if the spirit entity that has put the player into this situation users the voice of the London Underground to guide the player through the experience. So the player will hear instructions of what to do in the same tone and voice of the London underground announcer with of course some subtle story and mysterious undertones so that over time the story will develop and make more sense to the player leading to them hopefully being curious and having their own interpretation of what happened to them and why.

STORY-BOARDING

After coming up with my initial idea for the Designer Immersive Experience project I started to look into different Unity assets that i could use to design and create this experience. Looking through the unity asset store i found a train carriage model which i felt would work very well which was quite detailed in its design and was also a good length to be separated for the different puzzles for the experience.

Below is my initial level design layout for the experience where the user will start in the first train carriage with with multiple interactive objects around them which they can use to trigger the pressure pad next to the first exit. The next train carriage will be divided into two sections so the first will be a button sequence puzzle while the next will be a lever pull sequence puzzle. After completing these tasks the player will be able to get to the drivers cabin and meet the antagonist of the whole story.

During this weeks sessions we also had our first lectures on storyboarding for virtual reality and we were encouraged to try creating storyboards in a VR software such as tiltbrush, Quill, Microsoft Maquette etc.

These are some of my initial sketches and drawings that i created using Google Tiltbrush which i did initially find to be quite difficult to use due to my lack of experience using VR drawing software. I feel that this is something that I am going to need a lot more practice on to try and create a storyboard that isnt just visually appealing but also shows the story that i want to convey.

After trying to draw the full train model i started to isolate some of the storyboard drawings into individual scenes and the images above show the lever pull sequence of my experience. I did feel again that these drawings were not the best and i would need to rethink and practice how i would show these scenes in this storyboarding format. So i feel that the next steps for the DIE project will be to look at VR drawing tutorials and look at different storyboards for both video games, Films and VR experiences to better convey my story.