Materials required for this project

*Note: If you are only making a VR application, these items are all you need.

Player movement script: auto-walking and position reset

1. Auto-walking

After setting up my environment, I proceeded to make new player movement scripts using parts of what I wrote for my last milestone to better suit the VR platformer game I wanted to make. For this game, I settled for having player controls limited to the player moving their head/phone; due to time constraints, using a controller to control movements such as walking and jumping would have to be set aside because I did not have enough time to learn how to create a working UDP or TCP communication with the application (which would run the server program) and controller (which would run a client program that sends packets of information in TCP to the server) over wifi or other mediums.

I wanted to make my player walk in whatever direction they are looking at, except when looking too high or too low. For walking forward, I recycled code from my old movement script. Then I added public float variables that govern the minimum and maximum x rotation the main camera can be at that would allow for the player to move forward, and then if statements that would check the condition. Checking the condition actually turned out more tricky than I thought it would be, because the rotation values shown in the camera’s Inspector when testing my script are not shown as the values are stored in the code; the camera’s rotation is stored as a float equal to a hundredth of the value shown in the Inspector. Thus, when I wrote:

if(Camera.main.transform.rotation.x >= minXBounds  &&
Camera.main.transform.rotation.x <= maxXBounds)

I struggled for a while because I did not understand why my player would not stop moving when looking at an angle greater than my float maxXBounds. After using Debug.Log() to print debug lines to Unity’s console that told me what my main camera’s x rotation was during the game, I figured out that a simple fix was to just check for values in relation on minXBounds * 0.01 and maxXBounds * 0.01.

My auto-walking script can be found on GoogleDrive here: auto-walk script

2. Resetting player position when player is out of bounds

After I got this to work, I wanted to make game testing easier, so I needed a script that resets the player to the starting area when the player GameObject falls too far below the platforms (so I can test run the level without restarting the game). I found how to accomplish this in sluice’s answer at: saving and loading player positions. This also taught me that I can store float values and tag them with a string using PlayerPrefs.SetFloat(). The strings are used to refer to the variable I want to access with PlayerPrefs.GetFloat() later.

Second Milestone Overview

Progress Made

Making my first Unity 3D project

After completing Matthew Hallberg’s Instructables project, I went on to create my first Unity project. I started off by downloading and installing Unity 3D, a game development platform used for developing 3D and 2D games. Then, I created a new 3D project and learned how to perform basic workspace maneuvers such as panning and navigating the Scene View, and adding GameObjects to a scene. GameObjects are components in the game environment such as a plane for ground, and the player itself. These do not only encompass objects visible to the player, as Empty objects can be made. Scripts, or programs (on Unity, they are written in C# or Java script) can be attached to these to make the object manage functions such as ending the game when the player messes up. A scene is the workspace that objects can be placed in, and serves as a “level” in many games because the scene in play can be switched to allow a player to progress from one stage to another. The Camera object provides perspectives that the player can see the game through; first person games would have the main camera object within the player’s character and facing the direction the player object’s eyes would be facing.

Setting up the Unity project for a Virtual Reality supported Android build

Since I aimed to make a virtual reality game for Android, I learned that I should import the Google Virtual Reality Software Development Kit (GVR SDK) for Unity v1.60, which I downloaded here: GVR sdk download. The GoogleVR SDK is a Software Development Kit, or a bundle of  software tools used for the development of applications, directed at Virtual Reality applications. This SDK comes in a package that has prefabs, or already made GameObjects, that are free to use. The prefabs in this package have scripts attached to them that accomplish tasks such as making a camera object rotate using a phone’s sensors as an input.

After importing the package, I opened up the demo projects within the package to see if I could figure out how the prefabs worked by looking at how the projects were put together. Since multiple prefabs were used in the demos, I wanted to know which ones were the minimum needed to tie camera rotation to phone rotation, as that is all that I will need for my first VR game. After doing some research, I found out that I could not follow almost any guide on using prefabs from GVR SDK because a new version for the SDK had been released. To check the version and changes, I visited Google VR SDK release notes. Many of the prefabs people were using before had been deprecated, or removed, and replaced with new assets that accomplish the same function but in different ways. I found it difficult to find recent tutorials, but eventually found one on Youtube which takes a look at the demos on a version closer to the one I was using (v1.50) and how to set up the Unity project to build to Android: NurFace Games’ v1.50 release overview.

To set up the Unity project for Android, I navigated to File > Build Settings from the top tool bar of Unity and selected Android as my application’s platform and clicked Switch Platform. Then, to set it up for VR, I clicked on the Player Settings button next to “Switch Platform” and checked Virtual Reality Supported under Other Settings in the Player Settings tab that appears in the Inspector window. Then an area with the text “Virtual Reality SDKs will appear below it. Clicking the “+” symbol in this area will allow you to select from several SDKs; I selected Google Cardboard for this project.

Making a Move-able Player (Keyboard and Arduino NodeMCU control)

Moving on, I wanted to create a first person platformer as my first application. From NurfaceGame’s video (the one linked above), I learned that the GVREditorEmulator is the prefab used for creating the first person view with mobile input (rotation with phone movement). Adding a Camera object to the GVREditorEmulator in the inspector sets that Camera as the one that will rotate when the phone rotates. I then made a capsule GameObject as a simple player model and made a Camera (which I tagged as MainCamera) and the GVREditorEmulator as Child objects because they are components of the player. To assign the camera perspective that the GVREditorEmulator will use, I chose my MainCamera in the Camera input area in the GVREditorEmulator’s Inspector.

Making the player moveable with WASD and arrow key movement was my next goal. First, I made the plane in my game environment have a box collider so that the player would not fall through it. Then I wrote a simple script that changed the position of the character object by changing transform.position.x for the x-axis value, and so on if a certain button is down. This script was applied directly to the player object to make button presses change the player object’s position, not another object’s. I tested my code by putting my player on a textured plane so I could see if the player was moving relative to the ground. I found that the player object would tip over because it was a capsule object, so I froze its x, y, and z position and rotation in its Inspector- movement caused by a script changing the position of the object will still work.

Making the jumping script was similar to the walking script, making the player translate up on the y axis when the space bar is pressed with a short cooldown. However, since the jumping motion was updated every frame, my player jumped up extremely quickly. Furthermore, when the player landed on the plane, it tilted and flew away, which prompted me to lock its x, y, and z positions/rotations in its Inspector tab.

Establishing a Serial Communication between my NodeMCU and Unity 3D and making walking dependent on Camera orientation

After I successfully finished with these scripts, I wanted to make my NodeMCU controller control walking forward and jumping. I found a sample script that would allow me to send String to Unity from my NodeMCU through a serial communication in Farneze’s answer here: Arduino to Unity serial coms. I then added if statements to my walking code that made the player’s x position increase when my SerialPort object read a string equal to “walk” and added the same functionality to my jumping script. This ended up working, but I realized that I wanted the player to move in the direction he/she is facing, because they may not be facing the increasing side of the x axis. I found the right direction for writing my script in GargerathSunman’s answer on the Unity forums: Moving in direction camera is facing. I ended up changing the code to depend on a walkSpeed variable rather than my Time setting on Unity.

After implementing that logic, I found that when my player looked up and walked, they would “walk” up. Since I didn’t want this, I saved the player object’s y position as a float before moving and set the y position back to that after moving.

Adding moving platforms

The last thing I finished for this milestone is making several platforms that the player can jump over for a possible parkour simulator. For my horizontal and vertical movement looping scripts that I used for my platforms, I used KellyThomas’ script found here: platform movement loop.

Player movement script (C#) on Google Drive:

movement script with serial communication (reading) with Arduino

First Milestone Overview

Progress Made

For my first milestone of my VR game/controller project, I focused on making a simple working controller prototype and learn basic 3d modeling skills to use in making a 3d printed controller.

The first step I took was creating a schematic for the controller I was going to make. In order to do this, I had to first learn what connections were necessary between the buttons, NodeMCU dev board, and power source (battery pack). My first “schematic” was a sketch I drew on paper to familiarize myself with the basic design. I wanted to keep my first controller design very simple, so I settled for a rectangular prism design in my sketch. However, 3D printing the design is not something I am prioritizing and may not be included in this project as I will focus on the software and creating my own game after this milestone.

     Obviously, the sketch I drew (right) isn’t very clear, and exists mostly for my reference. Going forward, I looked for a program I could use to make a neater diagram that others could more easily understand. I cycled through several programs before settling on Fritzing, which allows users to select electrical components from an extensive library and drag them onto typical breadboard and schematic setups to make circuit designs quickly. However, I realized that the NodeMCU development kit model from Fritzing’s default library is not the same as the one I planned to use for my controller (NodeMCU 1.0 ESP-12E). I then learned how I could import new parts to use in Fritzing, and eventually found the model I needed at https://github.com/squix78/esp8266-fritzing-parts/tree/master/nodemcu-v1.0.

     With this, I had only to add my two push buttons and battery pack to the schematic and wire them to the development board. I hit another small road bump when I was choosing my button parts, as there seemed to be many four pin/terminal push buttons in Fritzing’s part library and I had no idea which one was the kind I wanted. This prompted me to do a lot more research about how my buttons work and the types of buttons there are before I could make a confident decision (I explain the specifics of what I learned in a section below).

     After the buttons were placed, the schematic was completed quickly and I got started with prototyping soon after. I tried making all the connections for the controller exactly as they would be in the printable controller model I would later design, but I realized that without the jumper wires and buttons actually being held in place by the controller shell, the wires would be troublesome to deal with because the pins of the buttons I used fit loosely in the female end of them, and could easily detach while testing. With this in mind, I decided to make my prototype on a breadboard and made a breadboard diagram on Fritzing using the same parts as in my schematic (right).

     My next step was to learn how to actually use a breadboard. This took me a while, because even after I did my research on how current runs through the board, I made many small mistakes in wiring the buttons. Mostly, these were trivial missteps like having my jumper wires in the wrong holes, which caused me some frustration later when I uploaded an Arduino sketch to the NodeMCU and began testing. I also learned that some parts of the NodeMCU can get very hot when it is being powered, as I touched one of the metal parts of it during the process (it was not hot enough to burn yet, though).

     Once I thought my connections were made correctly, I set up my Arduino IDE to have the NodeMCU 1.0 ESP-12E as a selectable board, and uploaded a sketch that enables the NodeMCU to communicate with a phone through a wifi network and makes the microcontroller in the development board respond to the signals from button presses by issuing a fire/move command to the game. I then launched the game on my phone to test the buttons, but they weren’t doing anything. This problem could have been with my wiring and hardware, or with the microcontroller not being able to communicate information to my phone over wifi.

     To diagnose this issue, I added a few test lines in the methods that set the firing and walking conditions to true/false to see whether the button presses were even being detected and thus if my wiring was correct. I learned that instead of using System.out.println(), I could use Serial.println() and see my test message output in the Serial Monitor on the Arduino IDE, and later that I had to set the Serial Monitor to the same baud rate as my IDE’s upload speed setting to see the string that I want to appear in the monitor.

     After I set this up, I found that my buttons were actually working as my firing and walking test messages were appearing. This led me to focus on what could be causing my microcontroller to not be able to communicate with the game, and I realized the wifi I was using was a school’s wifi and required an email address. Since the sketch I uploaded to the microcontroller accesses wifi only with the wifi name, IP address, and password, I tried my phone’s wifi hotspot instead. While my own hotspot did not work (the ESP-8266 could connect to it but could not communicate with the game over the wifi), the game worked perfectly when I tried others’ hotspots.

How it Works

The Development Board and Arduino IDE

The prototype I created is made of just a few major components: a NodeMCU 1.0 ESP8266-12E, two four-terminal push buttons, male to male jumper wires, and a breadboard. The NodeMCU is a development board with a microcontroller and an ESP8266 wifi chip in it. A microcontroller is a computer that can store memory and is used for a specific function which can be changed by uploading programs to it from a computer via a usb cable. The wifi chip allows the microcontroller to connect to wifi and therefore communicate with other devices. In addition to these parts, the development board has pins on it that can receive current from a battery and be used to connect the current to other electrical components such as the buttons used in this project. The Vin pin is what the wire from the positive terminal of a battery (usually this is the red wire) and is the terminal that can receive the highest current level on the board. Meanwhile, the Gnd pin is the ground pin that functions as a reference for what we will call zero volts of current (the black wire from a battery pack connects to this pin). A 3.7V Li-Po, or lithium-ion polymer battery is used to power my controller. The NodeMCU should only be able to handle up to 10V of current.

The program that my microcontroller uses to send information to the VR game based on button presses is what is known as a sketch, or a program written on the Arduino IDE. When uploading the sketch to the NodeMCU with a cable, a battery should not be used as the cable provides power to the board. When the board is connected, the Serial Monitor in the Arduino IDE can be used for testing and debugging purposes. It is a console where information and string from Serial.println() calls will appear; by putting the println() calls in methods that are called when a controller button is pressed, whether or not the println() message appears on the Serial Monitor when the button is being tested shows whether the button is wired correctly or not.

Buttons and the Breadboard

The push buttons used for my controller have four pins or terminals. Buttons function by completing circuit(s) inside when they are actuated, or triggered. A program or electrical component may be set up to react a certain way when the circuit(s) is/are complete, such as a light turning on when a light switch is flipped. The more technical term for these kind of buttons is momentary switch, which means that the circuit is being completed only when the button is being actuated; toggle buttons are an example of maintained switches, which complete or disconnect a circuit until the next time they are actuated. The type of momentary button used for the VR controller is double pole single throw (DPST), meaning that there are two circuits (double pole) in each of these type of buttons that are completed when actuated and a single orientation or way (single throw) in which they can be wired for the circuit to be successfully complete when actuated. For these buttons, their terminals must be connected to two jumper wires (female to female without a breadboard, male to male with a breadboard) on the same side; the other two terminals are there for mechanical stability, and connecting them to a circuit as well would cause a short circuit.

Because the buttons used are DPST, two of the terminals are not inserted into the breadboard. The two terminals inserted into the board are each wired to a gnd pin and either the D1 or D2 pin of the NodeMCU- the sketch executes methods to inform the game that firing or walking should be activated when pin D1 and/or D2 are receiving signals from the buttons. The breadboard is a type of board often used for testing electrical circuitry, as many common electrical components can be wired to the board without soldering. Current runs vertically (along the length of the board) through the power rails, or the two lanes on either side of the length of the board. Battery packs and power supplies may be wired there to supply power to the board. The middle two areas on the board has terminal strips, which are the horizontal rows of holes through which current runs horizontally. Separating the two middle areas is a trench. No current flows from one side of the trench to the other without jumper wires.

My starter project is the MiniPOV4, which is a device that uses LEDs to show a desired image that can be seen when the MiniPOV4 is waved quickly from side to side.