Third Eye for the Visually Impaired
The Third Eye for the Visually Impaired is a wearable device that uses ultrasonic sensors to emit and receive sound waves and alert the user of nearby objects via a buzzer or vibrating motor and can be controlled from a smartphone via a custom-built Android application.
Engineer
Diego B
Area of Interest
Mathematics and Electrical Engineering
School
Saint Ignatius College Preparatory
Grade
Incoming Senior
First Milestone
For my First Milestone, I enabled the buzzer and vibrating motor to function in conjunction with the distance recorded from an ultrasonic sensor. Before getting started, I looked into the functions of a Single Pole Double Throw (SPDT) switch which has three prongs, one to keep a constant connection with an input pin and two to switch between two different outputs—I’ll be using the SPDT switch to toggle between the buzzer and vibrating motor. I also researched the capabilities of an ultrasonic sensor, which, similar to a dolphin’s echolocation, sends out sound waves and tracks how long it takes for the emitted sound wave to return, giving the user an exact idea of how far their target is. Having a comprehensive understanding of the unique instruments that make up my project certainly made it easier to grasp and tackle various challenges.
Once I had also familiarized myself with a breadboard and Arduino Uno, I assembled the circuitry and began testing code. Since I came into Bluestamp with minimal coding experience, I had to research several Arduino functions and their utilities in order to correctly implement them into my code. My final code for this milestone featured six different ‘if statements’, or domains, to gradually adjust the rate at which the buzzer/vibrating motor operate depending on how far an object is from the ultrasonic sensor. These ‘if statements’ work by telling the buzzer or vibrating motor to turn on and off with a specific delay based on the distances recorded by the ultrasonic sensor. For example, the first ‘if statement’ in my code says that if an object is less than or equal to 12 centimeters away, the buzzer or vibrating motor will turn on for 1/20 of a second and then turn off for 1/10 of a second. This process repeats itself in a loop if and only if the distance is less than or equal to 12 centimeters. There are 5 more of these commands programmed for varying distances, and as an object moves farther away from the ultrasonic sensor, the buzzing or vibrating will slow down accordingly up to 100 centimeters.
The domain is as follows:
distance(cm) ∈ (0, 12] ∪ (12, 22] ∪ (22, 30] ∪ (30, 60] ∪ (60, 100]
This milestone was filled with many unique challenges and learning curves; however, getting acclimated to the tools and various devices at my disposal will certainly help me tackle a plethora of different challenges as my project progresses.
Second Milestone
Fritzing Diagram
MIT App Inventor Code
The three elements of my MIT App Inventor code can be seen here. Starting on the left and moving to the top row (3 blocks total) are the blocks that allow for bluetooth connection. These 3 blocks of code allow the user to connect to or disconnect from any nearby bluetooth devices. The bottom two blocks are responsible for turning the buzzer and vibrating motor on/off, as well as sending messages to the Arduino in the form of [B1/B0/V1/V0], which activate the buzzer/vibrating motor accordingly.
My goal for my second milestone was to establish bluetooth connection and allow a smartphone to control the Third Eye device. My first task was to connect the bluetooth module to my breadboard and Arduino Uno––doing so would allow the Third Eye device to connect to any bluetooth compatible device in the area. The bluetooth module has 4 pins: GND (ground), VCC (3.6v-6v), TXD, and RXD. The ground and VCC pins can connect directly to the ground and 5 volt pins on the Arduino, which, in my case, I set to the positive and negative pins on my breadboard. I then connected the TXD pin, which transmits serial data, to pin 2 on the Arduino, and I connected the RXD pin, which receives serial data, to pin 3 on the Arduino. Something that I had to keep in mind was that the RXD pin on the bluetooth module can only take up to 3.3 volts from the Arduino. Initially, I thought connecting the 3.3v pin on the Arduino directly to the RXD pin would satisfy the strict 3.3 volt intake, so I went with that and ran some tests.
In order to verify that communication has been established between the Arduino and bluetooth module, the user should be able to type “AT” into the serial monitor and receive an “OK” message from the bluetooth module. Upon testing for communication, however, I quickly realized that my “AT” messages were not being read. After screening a myriad of different code configurations, doing extensive research into the HC-06 bluetooth module, and even switching out the bluetooth module on my breadboard, I decided to create a voltage divider on my breadboard using a 430 Ohm resistor and a 330 Ohm resistor, in order to ensure that no more than 3.3 volts were connected to the RXD pin. Once I correctly wired my voltage divider and arranged the rest of the circuit, I finally received the infamous “OK” message on the serial monitor.
Now that the HC-06 module was fully operational, I began create a working app that can switch between the buzzer and vibrating motor––essentially turning the SPDT switch into a digital button that can be controlled from a smartphone via bluetooth connection. Using MIT App Inventor, I created a smartphone display which features three buttons: one to connect the smartphone to the HC-06 bluetooth module, and two to turn on/off the buzzer and vibrating motor. My code, still in MIT App Inventor, has three main elements which correspond to the buttons listed above. The first element––made up of 3 blocks of code––handles the bluetooth connection. First, it creates a list of available bluetooth devices that the user can connect to. If connection with one of the devices is successful, the on-screen button will display “connected”. Conversely, if the connection fails, the user will receive an error message and be prompted to try again. Lastly, if the user wishes to disconnect from a device, they can simply tap the blue button that reads “connected” and follow the on-screen prompt to disconnect. The second and third elements of my code––made up of 2 blocks–– are nearly identical and correspond directly to the on-screen Buzzer and Vibrator buttons. The 2 blocks say that if bluetooth connection is active, pressing either of the buttons will send a message to the Arduino and turn the on-screen color of the button to green and print “on”––the button will remain green and “on” unless the user presses the button again, in which case the on-screen color will turn to red and print “off”. Now, if there is no bluetooth connection, pressing either of the buttons will display an error message, informing the user that there is no bluetooth device connected. Once I had my app up and running, I began implementing the commands being sent from the smartphone app into my Arduino code, which controls the ultrasonic sensor and frequency of the buzzer/vibrating motor.
This was arguably the most difficult part of my second milestone. Considering I had minimal coding experience, I had to do a lot of research and troubleshooting before coming up with a working code. The first part of integrating the two codes was dividing the domains that dictate when the buzzer/vibrating motor would go off. Previously, the buzzer and vibrating motor would sound simultaneously. But since I wanted to activate them independently, I had to split the domains, creating 2 sections of code––one section for the buzzer and one for the vibrating motor. Once I had two, dissociated sections of code, I added a set of ‘if statements’ that activate the sections independently, based on if the Arduino receives B1/B0 or V1/V0 (B = buzzer, V = vibrating motor, 1 = on, 0 = off). After uncovering a few typos and carefully proofing the code, I connected the smartphone via bluetooth and assured that the app was running smoothly.
Third Milestone
Perforated Circuit Board with Arduino
Final Build
My goal for my third and final milestone was to turn the prototype of my Third Eye device, which was previously on a breadboard, into a fully functional, wearable device. To begin, I mapped out a design of my intended final product. I envisioned the perforated circuit board, with all of the components such as the ultrasonic sensor, buzzer, and vibrating motor, velcroed onto a wrist brace, along with the Arduino and HC-06 module. After preparing and reviewing my plan, my first task was to solder the buzzer, vibrating motor, ultrasonic sensor, and HC-06 bluetooth module onto a perforated circuit board, although I planned on securing the HC-06 module on the outer side of the wrist, flush with the wrist brace. Regardless, the HC-06 module would still have to be soldered onto the perfboard. Soldering, which is the process of melting solder into/onto wires to create a permanent connection, was a very unique challenge that I certainly enjoyed; however, soldering can be very taxing, since it requires intense focus and considerable planning. Before getting started on my actual perfboard, I practiced a variety of soldering techniques on a separate perfboard. After harboring a few successful connections on my practice-perfboard, I was ready for the real deal.
I started by soldering the ultrasonic sensor onto the perfboard and then soldering 4 jumper wires onto the 4 pins of the ultrasonic sensor, which would connect directly into the GND, VCC, TRIG, and ECHO pins on my Arduino. Once the ultrasonic sensor was securely soldered, I moved throughout the perfboard, trying my hardest to replicate the circuits and connections from my breadboard. After soldering for what felt like many hours, I began feeling like a surgeon, carefully and precisely connecting delicate jumper wires with scolding metal through roughly 0.042 inch diameter holes––my pair of tweezers and forceps certainly added to the surgeon persona I was imagining.
After verifying that the soldered connections were functioning as intended, I attached strips of velcro onto the wrist brace and the device itself. The perfboard, Arduino, and HC-06 module were then strapped to the wrist brace using a total of 6 strips of velcro: 1 on the perfboard, 1 on the Arduino, and 1 on the HC-06 bluetooth module––and of course the counter parts to those pieces of velcro were placed accordingly on the wrist brace. Before slipping into the wrist brace, I tested all of the connections and confirmed that the Third Eye device was working flawlessly.
Ultrasonic Sensor Display
Ergonomics and human bipedalism
Understanding basic human tendencies and bipedalism was crucial to creating an ergonomic and effective device. Something that often goes unnoticed is how humans tend to sway their arms when they walk, a perplexing behavior since the arms have no obvious role in bipedal gait. Researchers have hypothesized a number of beneficial effects of arm swinging, for example improved stability, reduced vertical strain on the centre mass, and reduced vertical ground reaction force on the hips, ankles, and knees. Regardless of its purpose, though, arm swinging is a habit nearly all humans possess and should be taken into consideration when creating a wearable device.
In order to maximize the potential reach of the ultrasonic sensor while maintaining a seamless design, I attached the Third Eye device to a wrist brace, and placed the ultrasonic sensor at the base of metacarpophalangeal joints, just below the fingers. Since the metacarpophalangeal joints are near the lowest point on the arm, swinging and swaying while walking will have an even more dramatic effect on the range of the ultrasonic sensor. In addition, I deliberately placed the ultrasonic sensor on the top side of the hand so as to leave the user’s palm available and free of limitations.
Placing 8 Third Eye devices on the body, spread out across the ankles, knees, hands, and shoulders, would provide the most coverage.