Third Eye for the Visually Impaired

The Third Eye for the Visually Impaired is a device that uses an ultrasonic sensor to detect if there is an object in the way of a user. The device is powered by an arduino and is programmed to beep and vibrate faster, the closer that the user gets to the detected object. Additionally, there is a second setting that allows users to use color detection, which would help people who are visually impaired figure out what color an object is.

Engineer

Ella D

Area of Interest

Biomedical Engineering

School

Dublin High School

Grade

Incoming Senior

First Milestone

For my first milestone I wanted to create a circuit incorporating a lot of the components that I want to add to the Third Eye for the Visually Impaired, such as the buzzer, ultrasonic sensor, button, LED, and slide switch. To do this, I did research on all the components so that I knew how to code the components and how they should be wired onto the breadboard. Unfortunately when assembling the circuits, I learned that the main component that I wanted to use for this milestone, ultrasonic sensor, was broken so I worked on learning about other components while waiting for a new ultrasonic sensor to be delivered. For example, I was able to make the slide switch turn on different parts of the breadboard. When the switch was flipped on one side, the buzzer would beep at the same time that the LED would flash. I plan on using this for the Third Eye, by making the buzzer beep faster as the user is moving closer to an object. I also plan on using the LED so that others can see the person, who is visually impaired, using the device. When the switch was flipped to the other side, another LED would turn on and the button would be used to turn off the LED. I plan on using this other side of the switch to change the mode of the device, by having a color detection tool for people who may be color blind, using the RGB sensor. Throughout this milestone, I learned about how to use the Arduino and how there are an endless number of possibilities with it, because there are so many different components that could be added!

Code for first_milestone_circuit

const int switchPin = 3; //sets slide switch to arduino pin 3
const int led1Pin, buzzPin = 12; //sets first led and buzzer to arduino pin 12
const int led2Pin = 13; //sets second led and button to arduino pin 13
const int buttonPin = 2; // sets button to arduino pin 2

int buttonState = 0; // variable for reading the pushbutton status

void setup()
{
Serial.begin(9600);
pinMode(switchPin, INPUT); //initializes the switch as an input
pinMode(buzzPin, OUTPUT); // initializes buzzer as an output
pinMode(led1Pin,OUTPUT); // initialize the first LED pin as an output
pinMode(led2Pin,OUTPUT); // initialize the second LED pin as an output
pinMode(buttonPin, INPUT); // initialize the button as an input
}

void loop()
{
Serial.println(buttonState);
buttonState = digitalRead(buttonPin); //checks the state of the buttom
int switchVal; // variable to hold the switch state

switchVal = digitalRead(switchPin); //set the switch value to the switchVal variable

if(switchVal == HIGH) //if switchVal is HIGH blink first LED
{
digitalWrite(led1Pin,HIGH);
digitalWrite(led2Pin, LOW);
tone(buzzPin, 1000);      delay(500);
digitalWrite(led1Pin,LOW);
noTone(buzzPin);
delay(500);
}
else //if switchVal is LOW turn on second LED
{
if (buttonState == HIGH) { //if button is pressed, turn LED on
digitalWrite(led2Pin, HIGH);
} else { //if button is not pressed turn the LED off
digitalWrite(led2Pin, LOW);
}
}
}

Second Milestone

For my second milestone, I wanted to create a circuit using the ultrasonic sensor, buzzer, slide switch, vibration motor, RGB sensor, and LED to create two different settings for the device: object detection and color detection. To create the object detection mode, I used an ultrasonic sensor, buzzer, and vibration motor. The ultrasonic sensor is used to detect the distance between an object and the device, while the vibration motor and buzzer vibrate and beep based on the given distance. To do this, I used if/else statements which took different distance values to control the frequency that the vibration motor and buzzer are used. When you flip the switch, the color detection mode is turned on, which uses the LED and the RGB Sensor. To make the color detection mode work, I programmed the RGB sensor to take input RGB values from an object and to display the output values onto the LED. I plan on using this feature for my third milestone by using the RGB values that the sensor detects and converting them to the color name. Then, I would like to use a text to speech program so that the user, who is visually impaired, can hear the color of the object. Throughout this milestone, I learned a lot about how to program using if and else statements to control different sensors. I also learned about how to use the RGB sensor and how to program it to show the color on the LED. 

Code for second_milestone_circuit

#include <Wire.h> //include the I2C library to communicate with the sensor
#include “Adafruit_TCS34725.h” //including the sensor library

const int trigPin = 10; //sets the trigPin to port 10 of the Arduino
const int echoPin = 9; //sets the echoPin to port 9 of the Arduino
const int switchPin = 7; //sets slide switch to arduino pin 3
const int buzzPin = 12; //sets first led and buzzer to arduino pin 12
const int vibration_motor = 4; //sets vibration motor to arduino pin 5
const int redpin = 3; //sets redpin to arduino pin 3
const int greenpin = 5; //sets greenpin to arduino pin 5
const int bluepin = 6; //sets bluepin to arduino pin 6

#define commonAnode false //sets the commonAnode LED to false

byte gammatable[256];

Adafruit_TCS34725 tcs = Adafruit_TCS34725(TCS34725_INTEGRATIONTIME_50MS, TCS34725_GAIN_4X); //creates an instance of the sensor

float duration; //long variable named duration
float distance; //gives an integer value for distance

void setup()
{
Serial.begin(9600); //starts serial comms @9600

Serial.println(“Color View Test!”); //prints “Color View Test”

if (tcs.begin()) {
Serial.println(“Found sensor”); //print “Found sensor” if sensor is found
} else {
Serial.println(“No TCS34725 found … check your connections”); //print if sensor is not found
while (1); // wait
}

pinMode(switchPin, INPUT); //initializes the switch as an input
pinMode(buzzPin, OUTPUT); // initializes buzzer as an output
pinMode(trigPin, OUTPUT);
pinMode(echoPin, INPUT);
pinMode(vibration_motor, OUTPUT);
digitalWrite(vibration_motor,LOW);
pinMode(redpin, OUTPUT); //sets redpin as an output
pinMode(greenpin, OUTPUT); //sets greenpin as an output
pinMode(bluepin, OUTPUT); //sets bluepin as an output

for (int i=0; i<256; i++) { //converts color values into more readable values
float x = i;
x /= 255;
x = pow(x, 2.5);
x *= 255;

if (commonAnode) {
gammatable[i] = 255 – x;
} else {
gammatable[i] = x;
}
Serial.println(gammatable[i]);
}
}

void loop()
{
long duration, distance;
Serial.println(buttonState);
int switchVal; // variable to hold the switch state

switchVal = digitalRead(switchPin); //set the switch value to the switchVal variable

uint16_t clear, red, green, blue; //declare variables for the color

if(switchVal == HIGH) //if switchVal is HIGH turn on ultrasonic sensor, buzzer, and vibration motor
{
analogWrite(redpin, 0); //light red led as per calculation
analogWrite(greenpin, 0); //light green led as per calculation
analogWrite(bluepin, 0); //light blue led as per calculation

digitalWrite(trigPin, LOW); //gets distance from ultrasonic sensor using trig and echo pin
delayMicroseconds(2);
digitalWrite(trigPin, HIGH);
delayMicroseconds(10);
digitalWrite(trigPin, LOW);
duration = pulseIn(echoPin, HIGH);
distance = (duration/2) / 29.1;

if (distance <= 3) { //checks distance and beeps/buzzes based on distance
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
}
else if (distance <= 10) {
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
delay(250);
noTone(buzzPin);
digitalWrite(vibration_motor, LOW);
delay(50);
}
else if (distance <= 20) {
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
delay(250);
noTone(buzzPin);
digitalWrite(vibration_motor, LOW);
delay(250);
}
else if (distance <= 30) {
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
delay(250);
noTone(buzzPin);
digitalWrite(vibration_motor, LOW);
delay(1500);
}
Serial.println(distance);
Serial.println(” cm”);

delay(500);
}
else //if switchVal is LOW turn rgb sensor side
{
tcs.setInterrupt(false); // turn on LED

delay(60); //wait for device to read

tcs.getRawData(&red, &green, &blue, &clear);

tcs.setInterrupt(true); //turn off LED

uint32_t sum = clear; // change values into hex code
float r, g, b;
r = red; r /= sum;
g = green; g /= sum;
b = blue; b /= sum;
r *= 256; g *= 256; b *= 256;

Serial.println();
Serial.print((int)r ); Serial.print(” “); Serial.print((int)g);Serial.print(” “); Serial.println((int)b );

analogWrite(redpin, gammatable[(int)r]); //light red led as per calculation
analogWrite(greenpin, gammatable[(int)g]); //light green led as per calculation
analogWrite(bluepin, gammatable[(int)b]); //light blue led as per calculation

if (((int)r) > 100){
Serial.print(“red”); //checks if the color is redish
}
else if (((int)g) > 100){
Serial.print(“green”); //checks if the color is greenish
}
else if (((int)b) > 100){
Serial.print(“blue”);//checks if the color is blueish
}

delay(1000);

}
}

Third Milestone

For my third milestone, I wanted to connect my device to a Color API so that the RGB values could be converted to the actual color name. Throughout this process, I realized that I needed to split my project in two parts, since particle photon would allow me to connect my project to the internet while the Arduino allowed me to have enough pins to attach all my components. I decided to make the object detection portion on the Arduino and the color detection portion on the particle photon. To get the RGB values to be converted to their color name, I used a web hook on the particle photon website, which sends data to the Color API website and receives data back in a specific format. This was definitely the most challenging part of my project, because I have never worked with APIs before. It was very interesting to learn about the different requirements when making a web hook, such as request type, query parameters, and a response template. Finally, after getting the Color API to work with my device, I used an LCD display to show the color name that the device retrieved from the website. If I had more time to work on this project, I would love to incorporate a speaker to this project, so that people who are visually impaired can hear the color name. This would require a text to speech API which would convert the color name to speech. Throughout this milestone, I learned that in the future, I would love to work with the internet of things, coding, and testing in the future. I think that Bluestamp gave me so much freedom to test out all of my interests and I am so glad that I applied to this program.

Code for third_milestone_arduino

const int trigPin = 10; //sets the trigPin to port 10 of the Arduino
const int echoPin = 9; //sets the echoPin to port 9 of the Arduino
const int switchPin = 7; //sets slide switch to arduino pin 3
const int buzzPin = 12; //sets first led and buzzer to arduino pin 12
const int vibration_motor = 4; //sets vibration motor to arduino pin 5

float duration; //long variable named duration
float distance; //gives an integer value for distance

void setup()
{
Serial.begin(9600); //starts serial monitor

pinMode(switchPin, INPUT); //initializes the switch as an input
pinMode(buzzPin, OUTPUT); // initializes buzzer as an output
pinMode(trigPin, OUTPUT); //initializes trig pin as an output
pinMode(echoPin, INPUT); //initializes echo pin as an input
pinMode(vibration_motor, OUTPUT); //initiallizes the vibration motor as an output
digitalWrite(vibration_motor,LOW); //sets vibration motor off
}

void loop()
{
long duration, distance; //creates duration and distance variables
int switchVal; // variable to hold the switch state

switchVal = digitalRead(switchPin); //set the switch value to the switchVal variable

if(switchVal == HIGH) //if switchVal is HIGH turn on ultrasonic sensor, buzzer, and vibration motor
{
digitalWrite(trigPin, LOW); //gets distance from ultrasonic sensor using trig and echo pin
delayMicroseconds(2);
digitalWrite(trigPin, HIGH);
delayMicroseconds(10);
digitalWrite(trigPin, LOW);
duration = pulseIn(echoPin, HIGH);
distance = (duration/2) / 29.1;

if (distance <= 3) { //checks distance and beeps/buzzes based on distance
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
}
else if (distance <= 10) {
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
delay(250);
noTone(buzzPin);
digitalWrite(vibration_motor, LOW);
delay(50);
}
else if (distance <= 20) {
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
delay(250);
noTone(buzzPin);
digitalWrite(vibration_motor, LOW);
delay(250);
}
else if (distance <= 30) {
tone(buzzPin, 1000);
digitalWrite(vibration_motor, HIGH);
delay(250);
noTone(buzzPin);
digitalWrite(vibration_motor, LOW);
delay(1500);
}
Serial.println(distance); //prints distance in serial monitor
Serial.println(” cm”); //prints cm

delay(500); //waits half a second
}

}

Code for third_milestone_particle_photon

#include <Wire.h> //include the I2C library to communicate with the sensor
#include “Adafruit_TCS34725.h” //including the sensor library

#define commonAnode false //sets the commonAnode LED to false

#include<LiquidCrystal_I2C_Spark.h> //include Liquid Crystal library for LCD screen

LiquidCrystal_I2C *lcd;

byte gammatable[256];

Adafruit_TCS34725 tcs = Adafruit_TCS34725(TCS34725_INTEGRATIONTIME_50MS, TCS34725_GAIN_4X); //creates an instance of the sensor

const int buttonPin = D2; // sets button to pin D2

int buttonState = 0; // variable for reading the pushbutton status

void setup() {
pinMode(buttonPin, INPUT); // initialize the button as an input

lcd = new LiquidCrystal_I2C(0x27, 16, 2); //turns on LCD display
lcd->init();
lcd->backlight();
lcd->clear();
lcd->print(“Color Name”); //prints color name onto screen

Particle.subscribe(“hook-response/color_name”, myHandler, MY_DEVICES); //subscribes to colorapi.com
Serial.begin(9600); //starts serial monitor
Serial.println(“Color View Test”); //prints “Color View Test”

if (tcs.begin()) {
Serial.println(“Found sensor”); //print “Found sensor” if sensor is found
} else {
Serial.println(“No TCS34725 found … check your connections”); //print if sensor is not found
while (1); // wait
}

for (int i=0; i<256; i++) { //converts color values into more readable values
float x = i;
x /= 255;
x = pow(x, 2.5);
x *= 255;

if (commonAnode) {
gammatable[i] = 255 – x;
} else {
gammatable[i] = x;
}
Serial.println(gammatable[i]);
}
}

void loop() {
uint16_t clear, red, green, blue; //declare variables for the color

tcs.setInterrupt(false); // turn on LED

delay(60); //wait for device to read

tcs.getRawData(&red, &green, &blue, &clear);

tcs.setInterrupt(true); //turn off LED

buttonState = digitalRead(buttonPin); //checks the state of the button

uint32_t sum = clear; // change values into hex code
float r, g, b;
r = red; r /= sum;
g = green; g /= sum;
b = blue; b /= sum;
r *= 256; g *= 256; b *= 256;

Serial.print(“\t”);

Serial.println();

String rgb = String((int)r) + “,” + String((int)g) + “,” + String((int)b); //creates string with rgb values
Serial.println(rgb); //prints rgb values to serial monitor

delay(1000); //wait 1 second

String data = String::format(“{ \”color\”: \”%i,%i,%i\” }”, (int)r, (int)g, (int)b); //sets format for data being sent as an rgb string
if (buttonState == HIGH) { //checks if button is pressed
Particle.publish(“color_name”, data , PRIVATE); //will send data to api if button is pressed
delay(10000); //wait 10 seconds
}

}

void myHandler(const char *event, const char *data) {
// Handle the integration response
Serial.println(data); //prints the data recieved
lcd->setCursor(0,1); //moves cursor
lcd->println(” “); //clears the bottom line
lcd->setCursor(0,1); //moves cursor
lcd->println(data); //prints data found
}

Demo Night

Reflection

Throughout my time at Bluestamp, my main goal was to understand which field of engineering I would be most interested in. I knew that biomedical engineering was very interesting to me, since it involved my two passions: helping others and problem solving through engineering. I decided to choose this project because it was a perfect example of what I could be doing in the field, since it involved using the Internet of Things to connect data to the internet, building, coding, and testing. I definitely took a lot from this program and I realized that I would love to pursue a future in Biomedical Engineering. I love how all the different types of sensors, code, and internet allow you to find an endless number of ways to help people! 

Start typing and press Enter to search

Bluestamp Engineering