Item Verification

The Problem

Have you ever wondered how factories handle huge amounts of product they manufacture every single day? That's done by using an LPN tag, which is a kind of license plate number for pallets that are automatically installed upon their creation. These LPNs store all the information about the pallet and the product that's on it. For example, at the Niagara Houston Plant, this includes the type of water bottle, the brand, the size of the bottle, etc.

However, there are problems with this system. Namely, only the very first pallet in each batch is checked by hand. After that, no one double-checks to make sure the tags and products still match. This is incredibly risky, as if even one tag is wrong, the wrong product could be shipped to the wrong customer. This can create problems that could potentially lose tens of thousands of dollars in sales every single day.

The solution I planned below is simple but powerful: a Computer Vision system that looks at every pallet, reads the LPN verification tag, and instantly confirms that the tag matches the product it’s attached to. Think of it like giving every pallet its own passport check before it leaves. This way, mistakes get caught automatically, shipments go to the right place, and costly mix-ups are avoided. Keep scrolling to see my journey in developing this software.

The Beginning

Accomplishments:

Learned the process of how Niagara bottles are made from initial plastic pellets all the way to the final pallet

Goals:

Install the camera in Houston for visual input
Grab the raw data and help use it to train the model

Houston Site Visit

Accomplishments:

Installed camera at Houston plants
Got plenty of images from my phone to use a data set for the model

Goals:

Check image data set and curate which ones would be best to help train the model with
Learn to access the OAK Camera and take notes on the library available to us

Annotation Era

Accomplishments:

Curated the data set so that we are using 70 images to train the model
Labeled each image by using annotations to show how certain objects look like

Goals:

Annotate the rest of the Houston set along with the 36 older images from the Mesa plant
Manipulate the images and allow the model to learn more use cases

Meeting the Model

Accomplishments:

Able to access and use unmodified libraries with OAK camera
Model has been trained with a completely annotated data set; it is currently not very accurate

Goals:

Edit the OAK camera libraries so that they are working effectively with our exact use case
Create a new workspace made specifically for polygons this time and upload all the images there
Create two separate models with one focused on zoomed in LPNs and one focused on zoomed out pallets

The New Model

Accomplishments:

Moved the annotated images into a better workspace so the model could accurately detect shapes
The new version of the model is significantly more accurate compared to the old one despite using the same image set

Goals:

Continue to test the new model across various brands and images to see its viability

QR Code Challenges

Accomplishments:

Fixed OAK Camera QR Code detection library so that it works for our use case
Discovered QR Code detection becomes more accurate if it has multiple attempts to try and recognize the QR code

Goals:

See the limitations of the loop solution and which QR codes are still unable to be scanned
Continue editing the OAK camera libraries so that they are working effectively with our exact use case

Code Detection Realized

Accomplishments:

Added barcode detection as another method of verification with Pyzbar
Tested Pyzbar and made sure that it works with still images
Found out that the Pyzbar library works with QR codes at a higher accuracy than the built in OAK Camera library

Goals:

See if it possible to move the camera closer, so that we can have more realistic testing for the Pyzbar library
Start editing new OAK camera libraries so that they are working effectively with our exact use case

Video Project.mp4

Major Victories

Annotated 106 images to train the Computer Vision Model

Implemented a QR Code and Barcode Scanner into the camera
Trained a new version of the Computer Vision Model that detects bottles and brand types

Primary Learnings

Constant communication is absolutely vital
Take frequent notes on what you're doing
Make a plan rather than jumping right into the work

Page updated

Google Sites

Report abuse