Ingredient Tracking

The Problem

Have you ever thought about what it takes to make sure a beverage tastes exactly the same every single time you drink it? At Niagara's KC2 Plant, that's a big challenge. This plant is unique because it focuses on beverages, rather than water, and batching along with aseptic processing is still relatively new. Right now, in the batching room, there are only a few checks in place to confirm that the right ingredients are being mixed in the right amounts.

That leaves a lot of room for error. If even one ingredient is off, the result could be a batch that doesn't meet standards, gallons of wasted product, and thousands in financial losses. Even worse, a mistake could reach customers and damage trust in the brand.

My solution changes the game. I designed a Computer Vision system that tracks both the type and the quantity of ingredients being added to every single batch through a video feed. The system then verifies that those inputs match the recipe perfectly. Think of it like having a digital recipe checker standing right next to the operator, alerting them to mistakes before they ever become a problem.

This creates an additional safety net for Niagara. It reduces waste, ensures consistent quality, and protects the company’s reputation. It is the kind of forward thinking technology that helps Niagara not just keep up with industry standards but set them. Keep scrolling to see my process in designing this tool.

My Journey

A New Start

The Task:

Look at the raw image data and making sure it matches the recipes, which are XML files, and build a program that automatically marks any inconsistencies.

The Plan:

Get the image data and the XML recipes to make sure that the program is actually checking what it needs to.

The XML file is ALWAYS right since it is the recipe, so grabbing the data from this accurately is the highest priority.

Make sure that image input can be accurately found. This requires OCR to read the text on the labels of the ingredients in the images.

Do the conversions between the image input to the names and units of measurement that is given in the recipe.

Compare the values from the image input and recipe values with the margin of error being an equivalent number of bags. This means there can be extra as long as it's less than one bag.

Output final results of the comparison and be sure that it is ready to present.

Recipe Retrieval

Accomplishments:

Step 1 is partially completed while Step 2 is fully completed
All XML data for the recipes has been given and the program is automatically able to scrape the useful values

Goals:

Get proper image data
Try and see how effective the OCR is on the image input, and what the limitations are

Image Inspection

Accomplishments:

Step 1 is now fully completed while Step 3 is currently being done
Researched into OCR and found multiple public libraries that have the ability to detect text on images
EasyOCR has been tested and is able to successfully grab the text in images, but with significant error

Goals:

Try and find the limitations of the other two OCR libraries, Tesseract and Amazon Textract

OCR Opposition

Accomplishments:

Step 3 is currently being worked on.
The results for both Tesseract and Amazon Textract weren't as effective as expected
Out of the three libraries, EasyOCR turned out the best, which is still less accurate than what is needed

Goals:

Try to see if there is any way that text can be read from the image more accurately

Start working on Step 4 of the project

EasyOCR Result

Tesseract Result

Paddle Power

Accomplishments:

Step 3 has been completed using another library called PaddleOCR
It is able to grab text pretty much perfectly for the use case

Goals:

Start working on the logic of the program to be able to compare the image and recipe text

Nicknames

Accomplishments:

Step 4 has been completed with the help of an Excel file
The program is now able to take the number code on the image and convert it to the name of the ingredient on the recipe
Any necessary weight conversions are also able to be processed now

Goals:

Perform the actual operations to check if the image values satisfy the recipe
Try and have a working prototype as soon as possible

Matchmaking

Accomplishments:

Step 5 has been partially completed while Step 6 is fully completed
The number codes on the ingredients are used to match the name and the weight is subtracted accordingly
All the data is now being outputted onto a locally hosted website

Goals:

Create a final output that shows if the image has enough ingredients to satisfy the recipe or not

Color Correction

Accomplishments:

Step 5 has now been fully completed
Visually outputs a color based on if there is enough of the ingredient.
Assuming that all ingredients are at acceptable quantities, the entire table becomes green to signify the recipe has been completed

Goals:

Pitch the software to a plant and collect feedback.

Major Victories

Successfully extracted recipe data from XML files

Accurately detected text from the image data set
Created a script that compares an image with a recipe file
Outputted the results on a locally hosted website for easy use

Primary Learnings

Understood how to effectively manage time when given a big task
Went through the process of developing a new solution that answers the given problem
Realized that I needed to fully understand the problem and everything it asked before developing the plan

Page updated

Google Sites

Report abuse