The Smart Refrigerator is a Raspberry Pi operated storage system that recognizes and manages the items in your inventory. Barcode-registered groceries are tracked by expiry date and usage.
About
Lily is a rising senior at Amador Valley High School. She is an aspiring creator who is passionate about climate change awareness, programming, and graphic design.
When people picture global warming, they think of gas cars, oil spills, and litter. What they don't really think about is all the food we throw away. Roughly forty percent of America's food gets tossed out. That's over 365 million pounds a day.
Accounting the land, water, and grain required to produce agricultural products (not to mention the exhaustion that comes from manufacturing, packaging, and transporting commercialized food products), the carbon footprint associated with wasted food soars above 3 billion tons of carbon dioxide.
A part of the problem is the way we store our produce. When stocking a fridge, consumers push back and hide soon-to-be expired products. When grocery shopping, customers often forget what they already have. To put simply, fridge design is meant for storage, not for display.
At my first hackathon, I developed an inventory app, Frigo, that acts as a "fridge on the go." The app notifies users when items in their fridge are about to expire. While my team and I made progress on the design, it was evident that requiring manual input from the user was inefficient and tedious. Here is where my project comes in. The smart refrigerator is designed to provide an effortless user experience:
Items are registered by barcode and weight.
Consumers can switch between an input and output setting to track the usage of their products.
Registered groceries are uploaded to a server and can be accessed through Frigo.
This outline will be explored and carried out in my three-milestone BlueStamp project.
Starter Project
With the intent to create a receipt scanner, I programmed an optical character recognition (OCR) script using OpenCV and Tesseract. After running an initial trial, the program had an output with over a fifty percent error. To improve the OCR's accuracy, I made changes to the source film and applied a binarization method to the Pi camera's footage.
After running the revised software, the percent error reduced to below a forty-percent margin, with most of the mistakes chalked up as misspelled words. To further improve the OCR, I had intentions to apply an auto-skew and spell-checker function to the program. However, taking a step back from this milestone, I realized that established grocery store franchises use heavy abbreviations to keep receipts concise. For example, Safeway's "Signature Select Whole Almonds" appear as "SIG SEL WHL ALMDS." Not to mention, receipt design varies among supermarkets. For these reasons, I decided to redirect my project to a more hardware-focused design.
Milestone One
After finalizing my project redesign, I programmed a Barcode Scanner using OpenCV and pyZbar. Having the receipt script to establish some familiarity with the IDE, I felt comfortable writing a script using object detection and the Pi camera. Using Adrian Rosebrock's "Real-time barcode and QR code reading with OpenCV" tutorial as a reference, I altered his code to fit my function. Rather than inputting barcodes one-by-one, Rosebrock's system runs live-footage and writes a CSV file for detected barcodes. He relies on a keyboard break to cut this live-footage. In writing his CSV file, Rosebrook appends a set to avoid duplicate barcodes. In my case, I choose to add another value, "Quantity" to increment with repeated barcode scans; after all, a smart refrigerator should track the usage of items. And, another feature particular to smart refrigerators is that there won't be a keyboard to break a while-true-loop. Instead, my loop breaks when a barcode is detected.
Milestone Two
With a working barcode scanner, I moved on to decorating my smart refrigerator with additional hardware. My second milestone included adding a slide switch, weight sensor, buzzer, and LED light.
Slide Switch
Three wires connect the slide switch to the Rasberry Pi: A red wire connects to a 5V pin (power), a blue wire, connects the middle switch pin to the Raspberry PI's GPIO pin, and a yellow wire connects to a ground pin. In my implementation, a toggle towards the red wire allows users to input items while a toggle towards the yellow wire outputs items.
Weight Sensor
Beginning with a pre-assembled HX711 weight sensor, I imported @tatobari's repository for the HX711 module. To calibrate the scale, I loaded a 411g can of chicken stock and tampered with the reference number until the sensor recorded the weight correctly. Tatobari's original software is an ongoing function that breaks with a keyboard trigger; it also takes multiple values before approaching the actual weight of the product. In my implementation, once the recorded weight on the scale passes 100 grams, the software begins a counter, and once that counter passes 10 ticks, the last recorded value becomes the weight.
Buzzer and LED Light
Using a breadboard, I connected the buzzer to the Raspberry Pi. In my application, when my software detects a barcode, the buzzer is triggered. Using a separate breadboard, I attached an LED light with a resistor and wires connected to power and GPIO pins. The LED light turns on and stays on during the runtime of my program. I included both hardware systems in my smart refrigerator so users would receive feedback while my software runs.
Milestone Three
For my final demo, I focused on creating a structure for my smart refrigerator. Using cardboard, double-sided tape, and hot glue, I built a make-shift storage system; the container is split into three subschemes. In the overhead compartment, I stored the Raspberry Pi along with the breadboard-buzzer system. The Raspberry Pi camera protrudes from the box and faces outwards. Behind the Raspberry Pi camera is the second overhead system: both the slide switch and the LED light cling to the roof of the box. Lastly, the weight sensor is hot glued to the bottom of the container, below a level platform.
