Draft Video 02 : One Minute Short ( 1st scene completed )

 

 YouTube Link

 

Experiment 01 has been completed. Planning for 3-4 experiment scenes.

 

YouTube VR setting

 

 

Testing with Google Cardboard

 

 Problem 01 : When user rotate view to the blank wall , he lost is sense of his positioning in the room.

Solution 01A : Use the shots, a further distance from the wall to include view of ceiling and floor. Instead of having a 12-movement loop cycle where the camera is placed too close to the wall , the shots will be re-selected and video re-edited to create the user experience of 4-movement loop cycle at the center of the room. Hopefully this will be less disorientating for him when he rotates. 

 

Solution 01B : Composite objects on the wall e.g. graffiti or pictures on the wall. This pose to be a problem because the graffiti needs to be composited and position accurately but separately for all 12 shots as the camera position for each shot is different. 

Application : Applying Teleportation to Theater Production ( not viable )

 Filming 360 video of  Theater Production

The ideal application from the VR experiment is to apply the stop motion camera movement technique into the filming of theater production. The theater brings the live experience happening in real-time to the audience seated at a distance from the stage. The more the audience pays , the closer the vantage point to the stage. 

With the new 360 VR experience, the audience can actually be on the stage and be transported point to point to different vantage point on the stage for different scenes. 

The cameraman will be missing from the shot for the audience to enjoy the full 360 view stage experience without breaking the immersion.

There are two possible 1st person continuity the camera can film in :

• The invisible standing audience watching the scene on stage and walking to a different location during a change of scene or a momentary pause or move towards the action.
  
  Eg. the 360 video can move into the intimate space in between the two actors when they whisper sweet nothing to each other. This facial closeup POV will be impossible without a 360 camera.
  
  
•  The invisible actor who participates in the scene. The single identity and continuity of the 1st person should be preserved and maintained. The 360 film should not switch multiple identities of multiple actors.   
  

Theater Production Research

The only way to do the research is to experience a theater production myself. After attending one , I realized that my stop motion camera technique will have challenges if I want to use this method for theater production. 

(1) Staging Design : the movement of and positioning of the actors are orientated  towards where the audience are seated. Especially for a small theater production , the audience are very close to the stage and have a good view of all happenings on the stage. There is no reason to move the camera on the stage and even there is one would require a revamp of staging design just for 360 video production.

(2) Stage Lighting : There are multiple lighting on the stage, that would cast a shadow on the camera and tripod that will  take a lot of effort to edit.

Conclusion : Due to the challenges mentioned above, it is not viable within the scope of NYP final project to film a theater production using the stop motion camera technique.

 

Test Video 01: 360 video upload to YouTube ( Go / No Go Gate )

YouTube Link

Prototype Testing ( Go / No Go Gate )

This is a short testing clip that is an important milestone. If the tests are successful , the project can continue else this project is not viable to produce and I have to discard it. 

Test 01 : OK Mouse rotation of 360 view is transferred to the next subsequent clip.

Test 02 : OK Works well in Insta360 studio and YouTube. Does not work well in VLC player.

Test 03 : OK Loop video test in Insta360 studio.

Improvement 01 : To remove tripod stand with Adobe Photoshop.

Improvement 02 : Balance color and brightness for all clips.

Improvement 03 : Extend the height of the tripod to the height of an adult eye level.

Purpose and Story

 Story

You are a test subject undergoing a series of VR experimentation. You are given a google cardboard to view the 360 content involving stop motion camera movement. You have signed the consent form.  Should you feel giddiness or nausea please stop viewing the VR head set immediately.

 

 Story Script ?

There is just a general story of creating a series of VR experimentation. At this point I don't have visibility of how the story ends , it can end with an application but I have no clue yet what and how to apply it.  I am designing the story on the fly and building it using an iterative approach (see picture below). What is being published that became of the story are the successful user experience felt by myself being the user. The problems might be in the draft versions and should be solved hopefully in the final publication of the 360 video. 

 

 

What is the Problem

Currently for still 360 image or video production, the user can only experience Three Degrees of Freedom (3DOF) with the google cardboard. He can rotate his head freely in all direction but cannot move forward , move backward,  move upward and move downward.  

 

Why does the viewer feel trapped when he experience the 360 VR?  

 

I think I do have a theory, I believe the 360 VR experience allows the viewer to experience the film in the 1st person perspective that the user feels the immersion that he exist inside the film ( 1st person ) and not "outside" (3rd person) .  Only in the 1st person perspective mode, the viewer is aware of his own presence in the 360 space. He realizes  that his movement is restrictive that causes his feeling of being trapped in the 360 space.

 

In the 3rd person point of view (POV), when we watch a non-360 film in the cinema we hardly feel trapped. The "I" (subject) am watching the screen(object) where the film is projected onto. Yes we do feel the emotions of the film , we empathize with the characters in the film but we know we are merely the spectator of and not the participant in the film. 

In the 1st person POV aspect, I think the VR games technology puts in more effort than VR 360 film by creating a virtual body for the user to occupy the 1st person. He can see and use the virtual body to interact with the VR object and VR environment in the game via hand controllers. The game industry is investing and inventing more technological gadgets eg. haptics suits and VR treadmills to bring the viewer immersion deeper into virtual reality. 

 

360 film production is still in my opinion old school , and seems disinterested in new technological gadgets instead rather focus on storytelling and cinematography. It is reasonable , the purpose of film and game are different. They do intersect with the common problem of the 1st person perspective mode , that the gaming industry has solved remarkably and continuously improve while the 360 film is still struggling with the 1st person.

 

In the typical 360 film production, the priorities are the storytelling and cinematography while the 1st person perspective plays the 2nd fiddle. The voice over (VO) narrator tells the story and the camera places the viewer into either the 1st or 3rd person POV depending on what works best for each scene and story. The continuity therefore is designed for the story flow, scene to scene, from  beginning to end.     

 

What is the Solution

The point-view experience is the user experiencing himself to be the center of his viewing experience. 

This point-view is being extended to an area-view experience where the user is transported or teleported point to point in an area space. He still maintains his 3DOF of the area space. The new virtual area-view experience is akin to a person taking one step forward to occupy a new position or location in reality. 

The 1st person continuity rather than a story flow approach to 360 filming disallow 3rd person POVs and jump cuts. Instead of jumping from scene, the area-view experience is employed to preserve the 1st person continuity. The 1st person user "walks" from one scene to the next. This also mean that the filming is limited to one location and in a controlled environment. Fortunately, filming a theater production in 360 provides one such environment.

New Problem

The new area-view experience may give the user deeper VR immersion but at the same time may cause motion sickness.   

Site Survey and Shoot

 2 Room Flat Shoot

The living room and bedroom is separated by a sliding door. The shoot location is in the bedroom.

A total of 48 shots are captured. 12 for each set of tripod height setting high, middle and low.

Final NYP Project Concept Proposal

Concept Proposal

This is an experimental 360 video project using the idea of stop motion technique being applied to the camera movement that would transport the viewer from point to point in a loop cycle.

Design Thinking : Prototyping

I visualize in my mind , in a step by step manner , the process of shooting each scene and how would the film output will turn out. 

The grid represents the floor tiles of an empty room. One 360 image is captured from the perspective of the 360 camera mounted on a tripod,  placed in the middle of one floor tile. The 16x shots from the 4x4 tiled room configuration are equidistance from one another can be rearranged in post production.   

The directional arrows shows the 360 camera movement. This scene gives a total of 12x shots , each 1 second long, piece together to form a 12 seconds looping video. 

The eye shows the initial viewing orientation of the 12x 360 images. 

I would imagine a potential issue that would throw this concept of mine out the window if the user decides to rotate his or her view ( blue eye ). The next image according to what my brain tells me, will rotate the user view back to the programmed initial viewing orientation ( yellow eye ). 

After testing a sample 360 video in YouTube , I am relieved that this is not the case, my fears are unfounded. The next scene actually follows the user's viewing  orientation contrary to my imagination.  More testing will be required.