Using IoT to Voice Control Most Anything (updated)

This instructable will show you how to use a small piece of software I developed called RTBobble loaded on an Adafruit Huzzah ESP8266 breakout board to do some easy IoT (Internet of Things) experiments. Hopefully this will be a useful tool for experimentally tinkering with WiFi controlled devices. We will also show how IFTTT and io.adafruit.com are combined to let you create custom audio commands to control the RTBobble using Google Home.

NOTE: These instructions have been updated since Google's interface to IFTTT has changed.

Here are just a few ideas of ways to use the Huzzah/RTBobble:

1. Turn on a light or sound a chime in your backroom when someone knocks on your front door.
2. Send a text to your phone if someone is trying to break into your house
3. Remotely activate a Halloween display
4. Remote control your phone's camera
5. Use IFTTT (If This, Then That) and Google Home (or other home automation device) to custom control anything including using voice control to start and stop a K'nex invention
6. Create a Bobble Meter that monitors high/low tide, temperature, weather (anywhere in the world), phase of the moon, or progress on a fundraiser
7. Turn on a light bulb when you've received an email or a donation, or someone is tweeting about you, or someone tags you in a Facebook photo
8. Control a Circuit Playground Express or other Arduino processor over WiFi
9. Automatically physically sound a gong when you get an online order
10. Feed your pet over WiFi
11. Make a clock that shows the number of minutes until the next bus arrives on your block

Detailed instructions on soldering headers to the Huzzah ESP8266 breakout can be found at: https://learn.adafruit.com/adafruit-huzzah-esp8266-breakout/assembly

1. Get the RTBobble source code on github at: https://github.com/reachandteach1/huzzahbobble and put it in a directory folder called huzzahbobble.
2. Setup Arduino IDE for the Huzzah ESP8266 breakout as described at: https://learn.adafruit.com/adafruit-huzzah-esp8266-breakout/using-arduino-ide
3. NOTE: You will either need a USB-serial cable or Adafruit CP2104 Friend USB to Serial Convertor (I personally recommend the CP2104)
4. Before compiling and loading the code into the Huzzah ESP8266, remember that you will need to first stick the ESP8266 into bootloader mode:
5. Hold down the GPIO0 button, the red LED will be lit
6. While holding down GPIO0, click the RESET button
7. Release RESET, then release GPIO0
8. When you release the RESET button, the red LED will be lit dimly, this means it's ready to bootload

You can supply power to the Huzzah ESP8266 in a few ways. Of course, you can use the cable or CP2104 board that you used to upload code into the chip, but that would be a kind of expensive waste of the hardware since you will probably want to keep this for other programming projects. You can use a 3 AA cell battery holder. The one I use has a power switch which is very convenient. Connect power to the board's V+ (not VBat) pin and connect ground to one of the two GND pins on the board.

You can also use an external 5V bench power supply. Finally, you can cut one end of a USB cable and add a couple of male crimp connectors to the Red and Black wires and you have an inexpensive USB power cable.

If you are going to use the Huzzah ESP8266 with an Adafruit Circuit Playground Express (see our instructable for RTPlayground2) which makes for a powerful combination of components, you can connect the VOUT of the Circuit Playground Express to the V+ pin of the Huzzah and connect GND pins from the two devices to each other. Powering up the Circuit Playground Express will now also power up the Huzzah ESP8266.

The RTBobble software uses MQTT (a lightweight communication system particularly useful for IoT applications). A FREE and conveniently available MQTT server can be accessed at https://io.adafruit.com . So before proceeding forward on this instructable, go ahead and create an account at https://accounts.adafruit.com/users/sign_up .

Now, login to your account at io.adafruit.com . The menu bar at the top of the web page should look like the photo. Click on My Key . It should bring up a popup box with your Username and Active Key (which is your Adafruit IO Key). This is kind of a long set of characters so highlight it and copy it into your clipboard for use in the next step. Remember to not share that key publicly otherwise others can use this username and key to hack into your feeds and dashboards.

If you are using this for the first time, you will need to configure the WiFi network access to your network. After applying power to the Huzzah/RTBobble.

Click on the Reset button (and release) and then click and hold the GPIO00 button for a couple of seconds.

You should see a network connection called BobbleConnectAP on your phone or laptop. Connect to that connection with the password "password". Your browser should automatically connect to a configuration screen. If not, open your browser and connect to 192.168.4.1.

Click configure WiFi, scan for your own WiFi network, and enter your network's password.

Enter your username and paste the AIO Key (Adafruit IO Key) which should be in your clipboard from the previous step.

For your feedname, specify "controldevice".

Submit these settings.

As long as you are using this device on the same wireless network, you don't have to do this again. If you want to reconfigure your Huzzah/RTBobble on another network or you want to change your feedname or AIO Key, just repeat this step.

You are nearly done... You still need to create the feed named "controldevice" at io.adafruit.com

In your io.adafruit.com account, go to the Feeds panel and click on +New Feed. Enter "controldevice" for the name.

Apply power to the Huzzah/RTBobble or click the Reset button if it is already powered up. The red LED should turn on indicating that the board has successfully connected to your WiFi network. The red LED should then turn off which indicates that a MQTT connection has been made to the "controldevice" data feed that you set up in the previous step.

Now select the controldevice data feed in your io.adafruit.com account. Click "+ Add Data" and enter "on" for the value. The red LED should turn on again. Click "+ Add Data" and enter "off" for the value. The red LED should turn off.

Congratulations! Your Huzzah/RTBobble is ready to use.

The RTBobble code fetches data from a feed called "controldevice" that you have configured in io.adafruit.com. The data will be one of the following: on, off, zap, or a number 0 to 180 . Based on this data, the Huzzah/RTBobble will respond in the following way:

•  on or off will set pin 16 high or low respectively (and turn the red LED on and off)
•  zap will set pin 16 high for 750ms and then go low
•  0 to 180 will set pin 13 to that angle using PWM
•  0 to 180 will set the output voltage of pin 15 to a scaled voltage between 0 and 1023 (0 to 3.3V)
•  0 to 59 will set pin 12 LOW and pin 14 HIGH
•  60 to 119 will set pin 12 HIGH and pin 14 HIGH
•  120 to 180 will set pin 12 HIGH and pin 14 LOW

*NEW FEATURE ADDED 9/30/2021* if you ground pin #5, zap will additionally cycle the servo attached to pin 13 between 0 and 90 degrees.

Now, let's start doing some cool stuff with the Huzzah/RTBobble. We'll continue to use the "+ Add Data" technique to control this board for test purposes. We'll talk about how to programmatically drive the board later.

Here's a simple motor circuit connected to the Huzzah/RTBobble. The numbers next to the components refer to component pin numbers on the 288+ Tronex from Tedco Toys. Sending "on" and "off" should now turn the motor on and off.

Now let's see how we can control the speed of the motor. Instead of connecting the input resistor to Huzzah Pin 16, let's try connecting to Huzzah Pin 15. This will map 0 to 180 to output voltage 0 to 3.3V. So, you can now enter values between 0 and 180 to vary the speed (but you will probably want to keep the numbers between 0 and 60 to not max out the motor driving voltage). You may notice an audio tone as you change the speed. The reason for this is that the output voltage from the Huzzah/RTBobble isn't an analog DC voltage but is in fact generated by changing the duty cycle of the digital output using PWM (Pulse Width Modulation). If you want to filter/smooth this voltage output, you can stick a 100uF capacitor between connection point 101 and ground. You will then stop hearing the audio tone.

It is really easy to control a 180 degree hobby servo with the Huzzah/RTBobble. Just connect the red servo wire to the V+ pin on the Huzzah, connect the brown wire to Gnd on the Huzzah, and connect the orange wire to Pin 13 on the Huzzah. The numbers 0 to 180 will set the servo to that number in degrees. Check out Step 7 of my instructable "Control K'nex with a mobile phone" to see how to modify a servo to work with K'nex.

With a L293D chip, you can control ON/OFF and the direction of an externally attached hobby motor. Wire up the circuit as shown. Note that pin 8 and pin 16 of the L293D are connected to each other as are pins 12 and 13. Huzzah pin 16 will turn the motor ON/OFF and is controlled with the on and off data commands. Pin 12 and Pin 14 will control the direction (forward and reverse) and can be set with 0 and 180 data command.

So up to now, we have been manually entering data into our io.adafruit.com data feed. The real power of working with the Huzzah/RTBobble is automatically populating the data feed from other data sources. That's what will make the list of examples that we listed at the beginning of this instructable possible.

One of the coolest and fun ways to do this is IFTTT (If This, Then That). You can get to this at ifttt.com and set up a free account. You will probably find that you will want to eventually subscribe for a paid account, but the free account is a great way to get started.

With IFTTT you can glue together communication mechanisms for a large number of service providers including Google Home, Facebook, Twitter, Adafruit as well as news and media services, weather feeds, and much more.

Its operation is very simple... you create a recipe that specifies what action you want to trigger upon and what you want to do when the trigger occurs.

Since I have a Google Home, I can create a recipe using Google Assistant for a trigger and with an action of sending data to my Adafruit data feed. For this example, I selected a trigger "Say a phrase with a text ingredient" "Activate Scene" from Google Assistant.

Set the scene to "motor on".

For my action, I selected "Send data to Adafruit IO" and specified the name of my feed (controldevice). Specify "on" for the Data to Save.

Add another trigger "Activate Scene" from Google Assistant.

Set the scene to "motor off" and specify "off" for "Send data to Adafruit IO"

Repeat for "motor forward" and "motor reverse"

After I save and activate this recipe, and whenever I'm near my Google Home, I can say "Hey Google, activate motor on" and "ON" is sent to my data feed and since my Huzzah/RTBobble is configured with this data feed, it will respond! In a similar way, I can say "Hey Google, activate motor off" and "OFF" is sent to my data feed.

With IFTTT, you can also create other triggers like "New mention of you" on Twitter or trigger on a change in weather conditions with Weather Underground or triggers from Facebook and much more.

Of course you don't have to use IFTTT to trigger your Huzzah/RTBobble. The real power comes in writing your own code in Javascript, PHP, or Python. You can interact with other API's out on the Internet or process JSON data sources from other research labs and public government data sources. Create your own data source tied to a fundraiser that you are passionate about. Build a "bobble meter" that can sit on your desk that shows your fundraiser's progress. Documentation to get started in programmatically sending data to your feed can be found at: https://io.adafruit.com/api/docs/#adafruit-io-http-api

One of the easiest ways to programmatically send data to your feed is by using a webhook. Webhooks allow a safe way to share a public URL that can send data to your controldevice feed. To create a webhook for your feed, simply click on Webhooks in your feed panel's sidebar, click the "+ CREATE" button in the popup, and click "CREATE" in the Create Webhook Receiver popup. You will see a URL that is the unique webhook for your feed.

Try putting the following code into an html file (with the url for the webhook inserted into the action string), open it in your browser, and click the "ZAP" button.

<html>
<body>
<form action="webhook url goes here" method="post">
    <button name="value" value="zap">ZAP</button>
</form>
</body>
</html>

Your feed should receive the word "zap"!

Below is a fuller featured ajax example:

<!DOCTYPE html>

 <html xmlns="http://www.w3.org/1999/xhtml">

<head>

    <title>Bobble Code calling io.adafruit</title>

    <script src="https://code.jquery.com/jquery-1.11.1.js"></script>
      <script>
         	  $(document).ready(function () {
		  $(".onoff").click (function setonoff() {
      		  var onoff=$('input[name=onoff]:checked').val();
      		  $('#thingvalue').val(onoff);
      		  $('#Save').trigger('click'); 
    		  });

		  $("#zap").click (function setzap() {
      		  var zap= $('#zap').val();
      		  $('#thingvalue').val(zap);
      		  $('#Save').trigger('click'); 
    		  });

		  $(".direction").click (function setdirection() {
      		  var direction=$('input[name=direction]:checked').val();
      		  $('#thingvalue').val(direction);
      		  $('#Save').trigger('click'); 
    		  });

             $("#Save").click(function () {          
                 var formData=new Object();
                 formData.value= $('#thingvalue').val();
                  $.ajax({
                     url: "https://io.adafruit.com/api/v2/webhooks/feed/"+$('#webhook').val(),
                     type: 'POST',
                     dataType: 'json',
                     data: formData,
                     success: function (data, textStatus, xhr) {
                         console.log(data);
                     },
                     error: function (xhr, textStatus, errorThrown) {
                         console.log('Error in Operation');
                     }
                 });
             });
         });

    </script>

</head>

<body>

<h2>Sample 2 Bobble code at reachandteach.com</h2>
     <form id="form1">

        ThingValue :- <input type="text" name="thingvalue" id="thingvalue" /><p>
	https://io.adafruit.com/api/v2/webhooks/feed/ <input type="text" name="webhook" id="webhook" /><p>
	<input type="button" id="zap" value="zap" />
<br><br>
        <input class="onoff" type="radio" name="onoff" value="off" /> off<br/>
        <input class="onoff" type="radio" name="onoff" value="on" /> on<br/>
<p>
        <input class="direction" type="radio" name="direction" value="0" /> forward<br/>
        <input class="direction" type="radio" name="direction" value="180" />reverse<br/>
<br>
        <input type="button" id="Save" value="Save Data" />

    </form>
</body>
</html>

Attaching your Huzzah/RTBobble's digital output pins (and ground) to another Arduino like Adafruit's Circuit Playground Express will expand the power of your ability to wirelessly control things even further. Refer back to Step 3 of this instructable to see how to connect the Circuit Playground Express to power the Huzzah/RTBobble. You can check out my instructable "Computer Controlled Tinkering w/o Writing Code" for more ideas.

I hope you have found this instructable to be a starting point in having fun inventing things that you can control over the Internet.