FindyBot3000 - a Voice Controlled Organizer

As any maker can attest, over the years, one collects a lot of parts. Perhaps it all started with a few resistors and LEDs, a breadboard, and an Arduino. But soon there were motors and motor drivers, IR rangefinders and tactile switches. Accelerometers, microphones, heatshrink tubes, potentiometers, relays, hookup wire, all those things salvaged from that printer you took apart... and the list goes on.

Now, organizing such a collection of goodies is no easy task. Some may have turned to baggies, for others, boxes. And then there were those who filled shelves with boxes, filled with baggies, filled with parts. All intricately labelled, yet still so cumbersome to manage.

This Instructable takes a stab at solving the age-old problem of organizing a bunch of things. So without further ado, FindyBot3000.

'nuff said.

This project is an Internet-of-Things organizer, designed for hands-free searching and intuitive voice-controlled management of parts.

This Instructable is organized as follows:

Overview - Describing how the tech is all linked

Prerequisites - Required Tools, Software, Electronics, and Parts Lists

Construction

• Mechanical - Building the organizer
• Electrical - Wiring it all up

Basic Testing

Software

• Download and setup necessary accounts
• Voice-Control - Linking Google Home to Particle Photon
• The Cloud - Deploying an Azure Function and SQL Database on Microsoft Azure

Future Plans - New features, animations, debugging techniques, etc.

A broad tech stack is used, ranging from lower-level C++ programming (Arduino-style!) to higher-level C# programming for the (Microsoft) Cloud!

Technologies and Languages

While many makers will be familiar with Google Home/Google Assistant, the other technologies may be unknown. Here's a quick summary:

Human - That's you!

Google Home - This is what you talk to, and turns voice into text. Behind the scenes, Google Home uses Google Assistant, which is really doing all the work. This project can actually work with any device that supports Google Assistant, like an Android phone or tablet.

IFTTT - If-this-then-that, a free online service which allows different apps to communicate. In this project, IFTTT is used to connect Google Home with Particle Photon.

Particle Photon - A cheap WiFi capable microcontroller, programmable Arduino-style! This chip drives the LED display on the organizer. Development with this microcontroller has us working with:

• C++ (Arduino style)
• Webhooks - Send/receive real-time data between applications (like Particle Photon to an Azure Function)
• Particle Dev - The integrated development environment (IDE) which is used to code for the Photon.

Azure Function - A piece of code that runs in the cloud. No virtual machines or web apps to worry about. This does all the heavy lifting in terms of processing text, building SQL queries, and any other work that would be much more difficult or (computationally) expensive to do on a microcontroller like the Photon.

SQL Database - Keeps track of all the parts: names, quantities, locations, search tags, dates, etc. The database is queried to insert, remove, find, or update parts.

In the next step, a sequence diagram shows how all these pieces fit together.

In the UML sequence diagram shown, the flow of each component is laid out. There's a ton of technology behind it all, so it's pretty incredible that FindyBot3000 can find parts within a second. This Instructable won't go into the nitty-gritty technical details of how each step is implemented. Instead, we will focus on building the physical assembly, deploying the necessary software components, and configuring everything to work together.

Now this is a pretty complicated drawing, showing how all the software components talk to each other, and in which order. Follow the arrows, noting which lifelines (vertical lines) they are pointing to, to get a sense of what is happening and when.

You may be wondering -- why don't we send the data from Google Home directly to the Azure Function, instead of first to the Photon? The reason is to provide a single source of debugging information. By logging all commands which are incoming and outgoing from the Photon, we can easily identify at which step in the sequence data loss or a bug may have occurred.

We will come back to this diagram in later steps, but for now, on to construction!

Tools:

• Jigsaw w/ fine wood blade
• Drill with 1/16th bit, 1/4" bit, and countersink bit
• Screwdriver
• Dremel w/ mutipurpose cutting bit
• Soldering Iron
• Solder
• Scissors
• Retractable utility knife
• Duct tape
• Double-sided foam tape
• Clamps
• Super glue
• Sand paper
• Measuring tape
• Carpenter/rafter/combination square

Hardware:

Electronics:

Accounts needed: You'll need to create an account with each of these services

• Google Account
• IFTTT
• Github
• Microsoft Azure
• Particle

Software:

• (Necessary) Particle Dev - Write Arduino-style code to run on the Particle Photon
• (Optional) Visual studio, community edition - Modify the Azure Function and manually manipulate the SQL database
  
  • Extensions: Azure Development, .NET desktop development, Desktop development with C++, Data storage and processing
• (Optional) Postman - For unit testing. Sends HTTP Post requests to the Azure Function.

The short animation demonstrates how FindyBot3000 was assembled.

Fusion 360 was used to design the CAD model, animation, and drawings for FindyBot3000.

By the way, you can use Fusion 360 for FREE as a hobbyist, like I am doing for this project!

I couldn't find an easy way to upload the parts, so the table below provides links to them in the Fusion 360 cloud:

Image Part Assembly Foam Pad Vertical Board Horizontal Board Middle Board Lower Cabinet Upper Cabinet

Drawings:

Horizontal board drawing

Vertical board drawing

Middle board drawing

Foam pad drawing

FindyBot3000 bundles the four individual organizer boxes in a slim wooden frame, sandwiched between slices of foam pad. The foam pads offer an aesthetically pleasing way of concealing all the wiring for the LED strips, as you'll see in later steps.

The first order of business is cutting the foam pads to size, from which the board dimensions are measured. Just lay the foam on a markable surface, then the board on top of the foam, and use a utility knife to cut the foam to the width of the board. With the foam cut out, measure the height needed by propping it up against the organizer box. Use a carpenter square or end of a board to make a square cut.

Great, first step done!

Cut the bottom, middle and top boards to length. Layering the cabinets as shown in the first picture produces the results. Stack the horizontal boards and the cabinets to find the total height of the assembly, and use that to mark the length the vertical boards need to be.

Alternatively, you can use the technical drawings. I've put images of them in this step for convenience.

Begin by assembling the horizontal boards with the right vertical board. Remember to drill pilot and countersink holes to relieve strain on the end of the boards.

The horizontal supports for all the containers in the cabinet are recessed about a half inch behind the bezel. To route the LED strips along these horizontal supports, a drill is used to make holes in the bezel to thread the LED strips through. I used sandpaper and a dremel to tidy up the holes.

1. Take each of the 7 2-meter long LED strips, and find the middle. The contact terminals in the middle will have solder on them, making them different from all the rest. (There is solder on the terminals every 50cm on these LED strips). Use a pair of scissors to cut the solder pads in half. Do this for all LED strips, leaving you with 14 1-meter long lengths. Leave the paper backing on for now.

2. Critical Step: Find a LED strip length with a male-end connector, as shown. Note: the direction of the little arrows on the LED strip should point away from the connector.

3. From the top-left of the four-cabinet assembly, feed the led strips in as shown. This will produce a zig-zag pattern when connecting all the LEDs. Note: starting from the top, for every odd row the arrows should point to the right. For every even row, the arrows should point to the left.

Before we finish up the assembly, it's a good idea to make sure all the wiring is correct and the LEDs are working.

1. Setup the circuit as shown in the Fritzing diagram.

2. Open up Particle Dev, or use the Particle Web IDE, to upload HelloWorld.ino to your photon.

This test lights up four LEDs along the length of some of the strip, to test that it's working.

WARNING: Do NOT turn on more than 4 LEDs at once. You risk consuming more power than the USB connection can supply. We will address this in the next step.

You may be wondering, why have all those power wires? Do I really need them? Yes, for two reasons:

1. To support turning on several LEDs at once, in case the search result turns up multiple items

2. To provide consistent 5V to all LEDs, as there is a small voltage drop across each LED.

To see what happens if you don't apply power at regular intervals, watch this video:

Why have such a large power supply? Do we really need 60A?

Technically, no. But if you want to run the LEDs at full brightness, then yes. So why limit your options?

Power Calculations

How do we arrive at 60A?

The manufacturer mentions 18W/meter @ 60 pixels/meter in the datasheet, and by datasheet, I mean a bunch of text hastily scribbled down on the product page:

Each row on the organizer is exactly 1 meter, and there are 14 rows, so we have 14 meters of LED goodness.

14 meters at 100% brightness, means the total wattage of the strip is 18W/m*14m = 252W.

We can use the power equation: P = IV, to give us a rough estimate of current (Amps) needed. Rearranging:

Power, P, is measured in Watts; I = Amperes; V = Volts

I = P/V = 252Watts/5V = 50.4A.

The next smallest power supply down from 60A was 40A - too small to power all LEDs at full brightness. So 60A it is. It's a good idea to get a power supply that exceeds the power requirements of your system by some margin, a ~12% safety margin (60/50.4 - 1) in this case is sufficient.

But what happens if you don't know how many watts are used per meter of your LED strip? Well, you can estimate it using rough estimates for other LEDs you may have. Most standard 5mm LEDs, like the red ones shown in the demo video, draw around 20mA of current when running off of 5V. Using that information, we can extrapolate power requirements for an LED strip:

1. 20mA (milliamperes) is drawn per LED, that's 0.020A (1000mA = 1A)

2. RGB LEDs have three LEDs each: one red, blue, and green

3. Find the number of pixels per meter, in this case, there are 60 pixels per meter.

At 14 rows, with each row being 1 meter long, that leaves us with:

14 meters * 60 pixels/meter * 3 LEDs/pixel * 0.020A / LED = 50.4A

Hey great, that's the same Amp requirements as calculated in our previous equation!

We're not done with those foam pads just yet. In the first step, we cut them to size. Now we'll route cutouts and channels for all the wiring to run through. Align the pad to the wooden frame, and use a marker to mark the routes the wires take along the foam pads, then use a dremel to cut holes and channels for said wire to run through. Since we haven't fastened the cabinets to the wood yet, you can just slide them length wise along the frame to access both sides.

As you can see, I did a pretty quick and dirty job of this. But no matter, it never sees the light of day once everything is put together.

To aid with wiring, the cabinets were never fastened in place.

With the wiring done, and the foam pads in place, finally install the vertical board on the left.

Nudge the cabinets so they are flush with the front of the frame assembly, and then drill two holes on the bottom of each cabinet into the supporting wood plank below. Fasten the cabinets to the assembly with the 3/4" machine pan-head screws.

I found that over time, the foam pads applied a little too much lateral pressure to the cabinets. This resulted in the LED strips buckling off of the cabinet rows. To fix that, I used some wooden wedges to push the cabinets apart and pull the LED strips taught.

WARNING: HIGH VOLTAGE - In this step, we attach the beefy power supply up to all the LED strip power lines.

Do not, under any circumstances, wire up the power supply with the power cable plugged in.

1. Begin with testing that the power supply turns on. See the first photo. Then unplug the power supply cable.

2. Wire up only the first row of LEDs to the power supply

3. Disconnect the second and third row of LEDs

4. Repeat the test on Step 12: Quick Testing. If the LEDs work as expected, great! If not, make sure the 5V and GND power supply lines are wired correctly.

5. When the first row of LEDs is tested and working with the power supply, go on and wire up the remaining LED strip power lines. Repeat Step 12: Quick Testing, but make the following modification:

Before After

This time, the four-led pewpew that zig-zags across the display should maintain the same brightness the entire way through. Like so:

6. Once all the LEDs are working as intended, go on and cleanup the wiring. Use some double-sided foam tape to secure the power supply.

The particle photon drives two important pieces of FindyBot3000:

1. The Relay - this allows FindyBot3000 to operate in low-power mode, by turning on the LED power supply only when needed.

2. The LEDs - the Photon drives the LEDs through the voltage step-up converter.

In this step, we link together the Google Home and Particle Photon.

Prerequisites:

- Log into your Github account

- Download and install Particle Dev

- Download and install Github Desktop

Steps:

1. Navigate to the FindyBot3000 repo on Github: https://github.com/Inventor22/FindyBot3000

2. Clone the repo to your computer

3. Open Particle Dev, and load the project folder: FindyBot3000\Photon\FindyBot

3. Plug in the Photon to your computer via USB cable, and select your photon from the devices menu in Particle Dev

4. Flash the particle photon with the firmware

Before we can test to see if the photon is working with the flashed firmware, we need to setup Google Home to talk to the Particle Photon, through IFTTT.

To test the firmware we just uploaded to the Particle Photon, there are a few simple commands we can trigger on the Google Home.

First, we make an IFTTT applet that is triggered when we say "Ok Google, Welcome $", where $ is the name of the individual you would like to welcome.

As soon as the applet has deployed, you can begin testing your FindyBot3000!

Use the table above to create the remaining IFTTT applets.

Currently FindyBot3000 supports 10 commands which require a SQL database, and another 4 commands which run locally on the Photon.

Requires SQL Database:

FindItem - Find an item

FindTags - Find items matching provided tags, colored on a gradient according to number of tags matched

InsertItem - Insert a new item into the organizer

RemoveItem - Remove an existing item from the organizer

AddTags - Add tags to an existing object. Ex: "Add tags diode light emitting to green LEDs"

SetQuantity - Set the quantity of an existing item

UpdateQuantity - Update the quantity of an existing item

ShowAllBoxes - Light up all the boxes which have items in them with random colors

BundleWith - Bundle a new item with an existing item. Ex: "Bundle 8 blue LEDs with red LEDs"

HowMany - Show, in ASCII text on the display, the remaining quantity of an item.

Local Commands:

Welcome - Scroll text across the display, welcoming the specified individual

SetBrightness - Change the brightness of the LEDs

SetDisplay - Turn on and off the display

ChangeColors - Change the color gradient used by the FindTags command

A picture speaks a thousand words...

Navigate to the Microsoft Azure Portal and log in.

Here we deploy the Azure Function, and set up continuous integration through Github. In this way, when new changes are pushed to the forked repository you created, they will automatically update the Azure Function with the new bits.

In order for the Particle Photon to successfully send data to the Azure Function, it needs to know where to send the message. In this step, we setup a Webhook between the Photon and our Azure Function.

In this step we deploy the SQL Database, and provide access to the Azure Function via the sql_connection string.

So we've got the Photon talking to the Azure Function, and the Azure Function talking to the SQL Database. But We're not out of the woods just yet. In order for the SQL queries in the Azure Function to work correctly, they need to query SQL Tables. In this step, we create the necessary tables in the SQL Database.

Manually editing a SQL database using SQL queries can be a daunting, frustrating, and time consuming experience. If you need to manually edit a few entries in the database, there's a much easier option using Visual Studio -- No coding required!

First, download Visual Studio Community. You will need this to update the SQL Database.

Next, navigate to the Azure Portal, click 'All resources' on the left menu, and click the FindyBot3000 database:

Click through to launch the Database with Visual Studio:

This will launch Visual Studio, and bring up a prompt for the database password:

The database may take quite a few seconds to load. To see the data in the tables:

Do this for both the Tags and Items table. You should see the rows in the tables populate:

Editing the tables from here is as simple as clicking a cell and editing the value. In this way, it's easy to update the container an item is placed in by changing the row/column shown in the database; adding and deleting items; updating quantities; and a bunch more.

A quick review:

There are two relevant tables, Items and Tags. Items stores the item name, quantity, row, column, and other such info for the items themselves, including a unique identifier NameKey. Tags stores the associated tags for a given NameKey, with the requirement thatall tags must be a singularized lower-case word.

The NameKey column must be unique within the Items table, and must exist in the Items table before adding entries to the Tags table. NameKey should be lowercase and singular. For example, to insert 24 Orange LEDs, the NameKey would be 'orange led'. Relevant tags for 'orange led' may include: orange, led, light, diode, etc. All lower case and singular. Each tag will take a new row in the Tags table, all with the same NameKey of 'orange led'.

To Add an item to the database, first add a row to the Items table, then add as many relevant tags to the Tags table using the NameKey from the Items table row. To create a new row, begin by clicking any one of the NULL cells on the very bottom row.

All necessary columns must be populated with valid entries before a new row can be committed to the database. Uncommitted entries will have a small warning beside them, as shown when partially adding an entry for 'orange led' in this example:

If one of the columns has an invalid value, or remains NULL when a value is required, an error message will pop up:

To cancel adding a row, hit escape on the keyboard.

Adding the relevant tags looks like so:

To Delete an item, first the associated rows must be deleted from the Tags table. Find the NameKey column of the row to be deleted in the Items table, then first delete all rows in the Tags table with that NameKey. Then, delete the row in the Items table.

Visual Studio will pop up an error if the NameKey to be deleted in the Items table is still present in the Tags table:

Deleting a row is as easy as a button press:

The other two tables, Commands and HttpRequests will show raw data logged from the Azure Function, which may help debug malformed JSON input or bad speech-to-text translations from Google Assistant.

In case the 'Open in Visual Studio' link doesn't work in the previous step, the following steps outline how to manually connect to the SQL database.

Navigate to https://www.whatismyip.com/ and make note of your IPv4 address.

Open up the Azure Portal > All Resource > findybot3000sqlserver

Copy the Server Name, then click Show Firewall Settings

Open Visual Studio > View > Server Explorer > click Connect to Database icon

Choose Microsoft SQL Server

Enter Server Name, SQL Server Authentication, User name, and Password

Finally, enter the database name under 'Select or enter a database name'

To find the database name: Azure Portal > All Resources > FindyBot3000Database > 'Show database connection strings'. The database name will be after "Initial Catalog=".

There's a lot more that can be done!

In the future, there will be room for an LCD to display debug information, or animating the lights, beyond just text scrolling.

Just a quick update -- Looks like this Instructable has won the grand prize in the organization contest! Thanks for the votes everyone :)