For my second milestone, I got a Unity game to work with the NodeMCU controller by making Unity and Arduino communicate through a COM port and Serial print lines. The game is a basic shooter with both a top-down view as well as a virtual reality perspective to play from.

The first time I opened up the Google Virtual Reality package in Unity, I was overwhelmed by all of the prefabricated objects and their scripts, so I looked up guides to get me started online.

Unfortunately for me, current versions of Google Cardboard’s assets are not backwards compatible with many of their older versions, so I ended up using an outdated version of Google Cardboard’s GVRMain asset for the virtual reality view to follow a tutorial. I found the learning curve of development with the old versions of GoogleVR to be easier than the current one because there are fewer objects and their names are more intuitive to me. For example, GVRMain is used for the main VR camera in the Unity scene in the old asset package, while in the new one there are more objects that depend on one another or require controllers to implement.

After scripting the prefabricated objects that came in the tutorial package, Unity updated and all of my scripts stopped working. My prefabs turned invisible and some wouldn’t even respond when I clicked my mouse  or when I pressed the button connected to my NodeMCU to shoot a ninja star.

Due to time constraints, I had to download Unity 5.2.1f1 (the newest update is 2017.1.0) and redscript my ninja star launcher and Arduino to Unity communications.

The goal of the game is to stop the blobs from going across the red line at the left of the screen in the top-down view or behind the player in the virtual reality perspective. The top-down mode can run on both the computer and on an Android device. On the computer, the player can either click with a mouse or press the button connected to the NodeMCU wifi chip in the direction he/she wants to launch a ninja star to shoot. In the virtual reality view, the player has to turn his/her head in the direction he/she wants to shoot a ninja star. In the future, the controller will work with the Android version of the game by using the wifi chip (the NodeMCU) to shoot ninja stars.

The top-down view of the game, playable on the computer.

The VR-perspective of the game, playable on a mobile device with a Google Cardboard-like headset.

Creating a connection from Arduino to Unity was the most difficult part of the programming job because I kept over-complicating things. For example, I tried using the strcpy() function in Arduino, which copies strings from one array address and puts it into another one that is given to the method as a parameter. Eventually, I got frustrated and took a break to do some more research on sending strings from Arduino to Unity and I ended up finding a simple method using Arduino’s Serial to allow the hardware (Arduino and the button) and the software (Unity) of my project to communicate.

Arduino’s Serial monitor allows the programmer to see string readouts that he or she adds the the Arduino. This is useful for printing out measurements or statuses of sensors on the Arduino device. For my project, I had Serial print the button’s value, which I found out could be either 0 or 1, representing the unpressed and pressed states. To find out which one was which, I had to setup up the Unity side of things.

I built on the code I wrote following the guide by adding ReadLine() function call in the Update method. This assigns the string that Unity receives from Arduino to the string variable input. If the input is the string “0”, the state is pressed and Unity tells the game to create a ninja star. This means the player is shooting! On the other hand, if the button is not pressed and Arduino sends a “1” through Serial and the COM3 port, Unity does not do anything in regards to the ninja star launcher.

My intensive project is a virtual reality game that is configured to play with custom controllers using the NodeMCU. For my first milestone, I got a demo program to work with the NodeMCU controller.

To start the process of making my virtual reality game, I first inventoried everything I will be using for this project.

Above: Bill of materials for this DIY project.

These items can be bought from the following links:
1. https://www.amazon.com/Virtual-Reality-Cardboard-4-0-6-3inch-Smartphones/dp/B06XTJNYTV/ref=sr_1_1?ie=UTF8&qid=1495402070&sr=8-1&keywords=fiit+vr+2

2. https://www.amazon.com/gp/product/B010O1G1ES/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B010O1G1ES&linkCode=as2&tag=wwwwirebeingc-20&linkId=65830b1abd06bb6d395c7dcd1656cdb4

3. https://www.amazon.com/gp/product/B008DS1GY0/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B008DS1GY0&linkCode=as2&tag=wwwwirebeingc-20&linkId=c36bb2cab22bc99e2175716f53488836

4. https://www.amazon.com/gp/product/B00A6SOGC4/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B00A6SOGC4&linkCode=as2&tag=wwwwirebeingc-20&linkId=7de98bf05097912138e6215862891ea0

5. https://www.amazon.com/HIGHROCK-Female-2-54mm-Jumper-2x40pcs/dp/B00GSE2S98/ref=pd_sim_328_6?_encoding=UTF8&pd_rd_i=B00GSE2S98&pd_rd_r=XG0F4ESX166C97ED2K4D&pd_rd_w=rhqlU&pd_rd_wg=xlRnD&psc=1&refRID=XG0F4ESX166C97ED2K4D

6. https://www.amazon.com/gp/product/B00K9HG98G/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=B00K9HG98G&linkCode=as2&tag=wwwwirebeingc-20&linkId=f29948c4cf219e3148bbf525782eb044

Any mobile VR headset should work fine for this project, but the one I linked is what I am using for this project. Some cardboard, printer paper, and tape or a similar adhesive will also be necessary. A soldering iron may be helpful.

Instead of using cardboard to create the controller, the handle and paddle portions can be 3D printed. Below is a model of what the most basic type of controller that can run the demo would look like.

Above: TinkerCAD model containing all the parts that the basic controller should have. The two half-spheres on the cylindrical part of the controller are the trigger and walk buttons.

After taking stock of the materials, I created a schematic for what the connections for the controller would look like.

Schematic for how the NodeMCU should be connected to the battery and the buttons. Made using Fritzing.

While researching the NodeMCU, I learned that the “D#” pins on the NodeMCU are just like the general input/output pins on a micro-controllers; they can be connected to buttons to take inputs to control the NodeMCU, which in turn communicates with the phone through Wi-Fi. I also learned that the VIN pin stands for voltage-in – that’s where the red (positive) wire of the battery pack will be connected to. The GND or ground pin, on the other hand, is used as a reference point for differences in voltage.

Major Challenges:

Looking at the breadboard setup I have above, it seems very simple. Well… I blew up two NodeMCUs trying to get it to work on the breadboard. The first problem I had was with the buttons. I wasn’t paying attention and had all four of their pins plugged into the board, causing them to short-circuit.

After my first NodeMCU started smoking, I realized something was wrong and moved the buttons so that only 2 leads on the same side of each button are in the board. My second NodeMCU was fried because I connected 5 Volt wires to 3.3 Volt pins, meaning the current didn’t pass through the regulator and ended up destroying the Wi-Fi chip. While I was trying to figure out what went wrong I  learned that the standard USB to micro-USB cable transfers data and provides power at 5 Volts.

The last problem I ran into was with the internet at BlueStamp. Since there was a two-step login system for the Wi-Fi, it was difficult to get my wifi chip to connect to my phone. I solved this by using someone’s Wi-Fi hotspot and managed to get the demo working perfectly.

Challenges

Most of my setbacks were the result of not paying attention and being unfamiliar with the tools. The first difficulty I faced was with the long wire leads that I had to bend on the underside of the printed circuit board. Due to a messy soldering job, some of the solder and excess wire leads crossed and stuck together on the bottom of the PCB. I learned that this could lead to the current flowing to the wrong places and short-circuiting the mini-POV and potentially blow out the LEDs. I spent most of day two at BlueStamp cleaning the back of the printed circuit board and fixing the soldering.

Another issue I ran into was with the LEDs not working. I soldered and unsoldered each part multiple times. I managed to fix one of the LEDs this way because the the soldering had not gone into some of the PCB’s rings enough for the through-hole soldering to work. I must have broken the red and black wires at least three times each trying to fix them before realizing that the remaining LEDs themselves were broken, and that the soldering or wiring jobs were fine. Through this process, I learned that the four legs on the RGB LEDs each have their own functions; three of the leads are for each color (red, green, and blue), and the fourth one is a normal lead.

In retrospect, these obstacles taught me how to be a more independent problem solver and researcher. I learned how to keep the circuits from shorting out and to be careful with the leads so as to not break or overload them when I’m soldering components onto a PCB.

The Components

• The mini-POV is built on a printed circuit board (PCB), a layer of insulating material with conductive material – copper in this case – etched into its surface.
• In the middle of the PCB is a microcontroller or integrated circuit (IC). The main function of this component is to store the images and desired flashing patterns that are displayed by the LEDs. While I was researching each component, I learned that there are three types of integrated circuits. An IC can be an analog, digital, or mixed signal microcontroller. An analog microcontroller uses a continuously variable signal; it depends on using changes in frequency, total charge, or voltage as information. On the other hand, a digital microcontroller has two states to store data: a ground state and a state that is closer to the voltage supply. A mixed signal microcontroller has both a digital and an analog microcontroller.
• Two types of resistors were used for this project.
  • The ones that I soldered on first were the 2.2k Ohm resistors.
    • Three of these resistors were placed to the right of the transistors and directly above the microcontroller.
    • The 2.2k resistors use electrical interference to gate the current that is delivered to the transistors to prevent overloading the IC. The remaining 2.2k Ohm resistor was placed closer to the USB jack.
    • It plays an important role in uploading images because it lets the computer know when the USB port is connected and when an image is ready to be uploaded.
    • The second type of resistor I soldered onto the PCB is the 47 Ohm resistor. These are placed close to the LEDs to reduce the current flow going into the LEDs.
      • They control the brightness of the LEDs and make while making sure that they do not overload and stop working. Two of these were also soldered close to the USB jack.
• At the top right of the PCB are three capacitors.
  • Of the capacitors, two are electrolytic and one is ceramic.
    • The electrolytic capacitors filter out low frequency noise.
    • The ceramic capacitor filters out high frequency noise from the input and output voltages.
    • Together they keep the battery voltage smooth while reducing interference.
  • To the left of the capacitors is the potentiometer.
    • This component translates rotation into resistance and is used to set the speed at which the LEDs blink. Potentiometers are often used to set volume and frequency ranges in electronics.
• At the top left of the printed circuit boards are three transistors.
  • These components are strong enough to handle the 8 LEDs and can amplify signals 100 times.
• Below the microcontroller is a 12 MHz ceramic oscillator
  • It is used by the IC to keep track of time to flip through the LEDs consistently.
  • It does this 12 million times a second, hence the name.
• Zener diodes, were soldered directly to the left of the USB jack.
  • These are polar, unlike the non-polar resistors.
    • This means that the orientation they are soldered in matters, as they only allow current to go one way.
  • These semiconductors keep the voltage stable in the USB data lines.
• Of course, all of this needs power to work correctly. The PCB is connected to a battery pack via a red and a black wire that are soldered onto the board.