Instructables Contest Notifier - 3 Ways!

In this Instructable, I'll show you 3 different ways you can get notified about the upcoming Instructables Contests!

First I'll show you the basic heart of all the notification methods: A simple web server that does web scraping in Python. Then I'll show you how that web server's API can be used by 3 different devices:

• An Adafruit MagTag provides rotating messages about each contest.
• An Adafruit PyPortal ups the game with graphics from the Instructables website.
• Finally, an RGB matrix is interfaced with an Adafruit Matrix Portal M4, and it shows the date, time, weather, and information about upcoming Instructables contests.

With this information, you can duplicate these exact steps to build the same projects, or you can use what I'm providing as a jumping off point for your own ideas.

Best of luck to you and happy programming!

The different methods provided in this Instructable use different materials, but at a minimum you will need:

• A computer running Windows, MacOS, or Linux which is capable of running Python 3.
• One of the devices and supporting materials to display ongoing contest information.

In each step, I'll provide additional information about materials you'll need for each specific variation.

The heart of this project is the web server which runs on Python. Whether you're using a Windows desktop, a Mac laptop, or a Raspberry Pi, this step will give you some pointers for setting up the latest version of Python.

For Windows or Mac, head over to Python.org.

From a web browser, go to the Python.org website. From there download the latest Python 3 version that is available for your platform.

Note: If you already have Python installed on your computer, you can use it as long as it's Python 3.6 or higher.

For Raspberry Pi, if you are using the Raspbian OS, you should already have a version of Python 3 that will work.

Full disclosure: The web server part of this project has been tested on Window 10 and Mac OS 11.2. It has not been tested on Raspbian or any other versions of Windows or Mac. It SHOULD work, but knowing how things can be, no guarantees.

Now that you have Python, you'll need to get the project with the web server code. You'll also need to import Python modules that are needed to run the web server.

1. Download the Contest Scraper Python project from this GitHub repository.
   • If you're not familiar with Git or if you don't have it setup, just choose the "Download ZIP" option from the Code dropdown and then unzip into a directory of your choosing.
   • If you are familiar with Git and GitHub, choose your favorite method to grab the code.
   • Note that this repository includes the web server code as well as the other code to support the multiple devices mentioned in this Instructable.
2. If you're on Windows, open the Command Prompt. If you're on Mac or Rasp Pi, open the Terminal.
3. Change directory to where you unzipped the Contest Scraper project.
4. Install the pipenv tool using Python's pip command:
   

   pip install --user pipenv

   NOTE:If pip doesn't work in your Python 3 environment, you may need to use pip3 instead.

5. Now that pipenv is installed, run the following pipenv commands:
   

   pipenv install
   pipenv shell

6. At this point you should be in a Python virtual environment, and all the needed Python modules should be installed. Try running the web server using the following command:
   

   python app.py

   If you see the message "Running on http://0.0.0.0:5000/", your web server is running!

7. From your browser, try going to http://127.0.0.1:5000. After a brief delay you should see something similar to the picture above. This page allows you to see the Instructables contest information collected by the web server, and it will allow you to do some configuration as well to customize the notification behavior.
8. Check to make sure your web server can be accessed by other computers on your local network:
   • Run the appropriate command on your computer to get your IP address.
     • On Windows the command is "ipconfig".
     • On Mac the command is "ipconfig getifaddr en0" (wireless) or "ipconfig getifaddr en0" (wired).
     • On Raspian the command is "hostname -I" (capital "eye").
   • Check that port 5000 is open on your computer's firewall.
     • On this one, I'm going to punt to Google. Use Google to lookup "Windows firewall open port" or a similar query for your operating system.
   • From a computer on your network that is not your web server, open a browser and go to the URL:
     
     http://your_web_server_ip:5000/
     
     

     You should be able to see the same web page you saw when you tried it directly on your web server computer.

Additional Information

The web server provides a nice page for users to see the information collected, but the devices use an API (Application Programmer Interface) to get information. In your browser, you can see that information from these URLs:

• http://127.0.0.1:5000/api/v1/contests
• http://127.0.0.1:5000/api/v1/meta

The web server reaches back out to Instructables every two hours to get updated information. The devices that connect can use the information from the "meta" link to determine when they should get an update from the web server.

This project was developed using a Python virtual environment tool called pipenv.

• If you want to learn more about pipenv, you can find information here and here.
• Generally using a virtual environment is a best practice if a computer is being used for developing multiple Python projects. If you choose to use a different virtual environment tool or would rather just load the Python modules as part of your standard Python environment, reviewing the Pipfile will give you some guidance as to what Python modules are needed to run this project.

There are many Python frameworks for creating web servers. This project is using Flask.

• You can learn more about Flask here and here.
• I've set up this server to listen on port 5000 on all IP addresses the computer has. If you wish to change that behavior, you can change the last line in app.py to configure specific IPs and ports.
• Flask has the concept of Development mode and Production mode. Since this project is not "production grade", I've chosen to keep it using the built-in server which should be good enough for our purposes, but it is running in Production mode which reduces it's debugging capabilities. To find out more about other ways that Flask can be configured and deployed, you can review the web pages here and here.

What you need

• Adafruit MagTag
• USB C 5V Power Supply or other means to power it
• USB cable, either USB A to C or USB C to C depending on your computer.

How to do it

The MagTag is the simplest of the 3 devices that displays the Instructables Contest information. Here's the best way to get started:

1. If you've never worked with a MagTag before, review the information in the Adafruit Learn Guide here. You should run through their basic setup and make sure you can run their Internet test. Beyond that, it might be helpful to also see if you can run a simple example like this one.
2. With your USB cable, connect the MagTag to your computer. It should appear on your computer's desktop as a new drive with the name "CIRCUITPY". (If it doesn't, review the information on this page.)
3. Open two windows on your computer: One to the CIRCUITPY drive and one to the directory where you downloaded the web server project.
4. In the web server project directory, go into the devices folder, and then the MagTag folder.
5. Open the file "secrets_example.py" in a text editor:
   • Update information about your router in the "ssid" and "password" fields.
   • Update information about your web server in the "local_server" field.
   • Save the file as "secrets.py".
6. Make sure you have the CircuitPython libraries that you need. In the MagTag/lib folder, you will find a list of the library directories and files you need in the info.txt file. Information on installing the libraries can be found here.
7. Make sure your web server is running.
8. Copy all the code in the MagTag folder to the CIRCUITPY drive by dragging the files between windows.

After a minute or so, the MagTag should connect to your web server and start displaying information about the current Instructables contests!

Ever 10 seconds, the contest name should change to display a new contest.

How it works

• The MagTag reaches out to your web server to get contest information via the REST API. It stores an index of which contest it is displaying before it goes into "deep sleep" mode for 10 seconds.
• When the MagTag wakes up, it gets the index it stored in the previous wake cycle, increments it, and then displays the next contest information.

Troubleshooting

• If the MagTag displays "Contest Data Offline", that means the MagTag was unable to connect to your web server.
  • Make sure the web server is running.
  • Make sure you secrets.py file has the correct information about your router and web server.
  • Make sure the firewall on your computer isn't blocking external connections
• You may want to watch console messages on the MagTag to see if you can get more information.
  • The easiest way to work the console on these CircuitPython devices is to use the Mu editor.
  • Once you have downloaded and installed Mu, start up Mu, then connect your device. Click on the Serial or REPL button in the editor's tool bar. You should start seeing messages in the console at the bottom of the editor.
  • Ctrl-C will halt code execution. You can hit enter to go into the REPL. Ctrl-D will soft boot and restart code.py on the device.
  • You can load and save code directly on the device using the editor. Just make sure you also save any changes to your local computer for safe keeping.

Enhancements you might try

This project is using the barebones of the MagTag and is optimized for power savings. However you may want to enhance this project:

• There's no graphics being shown on this device. A simple background BMP might be a nice touch.
• The MagTag has 4 NeoPixels at the top and 4 buttons at the bottom. How might you use those?
• The CircuitPython Graphics library has a qrcode generator mentioned in this project. How might you use the qrcode feature?

What you need

• Adafruit PyPortal
• Micro SD Card, 4GB or larger
• MicroUSB 5V Power Supply
• Micro USB cable to connect the device to your computer

How to do it

Taking things up a level, the PyPortal shows the same information as the MagTag, but it also includes the graphic from the Instructables contest page for the specific contests. It makes for a bit more compelling display.

1. Again, you may need to familiarize yourself with the PyPortal and make sure it operates before attempting this project. Review the PyPortal Adafruit Learning Guide and make sure that it can connect to the Internet. The learning guide also includes this Quotes project. If you can get that up and running, you should have no trouble getting this project to work.
2. For this project, you will need to insert a FAT formatted Micro SD Card into the PyPortal.
3. With your USB cable, connect the PyPortal to your computer. It should appear on your computer's desktop as a new drive with the name "CIRCUITPY". (If it doesn't, review the information on this page.)
4. Open two windows on your computer: One to the CIRCUITPY drive and one to the directory where you downloaded the web server project.
5. In the web server project directory, go into the devices folder, and then the PyPortal folder.
6. Open the file "secrets_example.py" in a text editor:
   • Update information about your router in the "ssid" and "password" fields.
   • Update information about your web server in the "local_server" field.
   • Save the file as "secrets.py".
7. Make sure you have the CircuitPython libraries that you need. In the PyPortal/lib folder, you will find a list of the library directories and files you need in the info.txt file. Information on installing the libraries can be found here.
8. Make sure your web server is running.
9. Copy all the code in the PyPortal folder to the CIRCUITPY drive by dragging the files between windows.

After a minute or so, the PyPortal should connect to your web server and start displaying information about the current Instructables contests using fancy graphics!

How it works

• When the web server scrapes data from Instructables, it grabs the URL for the contest graphics and downloads them. However the graphics are not well formatted for the PyPortal. It uses a Python module called pillow to reformat the graphic into a format that works better for the PyPortal.
• The PyPortal tries to minimize the amount of networking it does by taking the graphics it gets from the local web server and storing it on the SD card. This caching speeds up the display updates after the graphics have been stored.

Troubleshooting

• If the PyPortal displays "Web Server Offline", that means the PyPortal was unable to connect to your web server.
  • Make sure the web server is running.
  • Make sure you secrets.py file has the correct information about your router and web server.
  • Make sure the firewall on your computer isn't blocking external connections
• You may want to watch console messages on the PyPortal to see if you can get more information.
  • The easiest way to work the console on these CircuitPython devices is to use the Mu editor.
  • Once you have downloaded and installed Mu, start up Mu, then connect your device. Click on the Serial or REPL button in the editor's tool bar. You should start seeing messages in the console at the bottom of the editor.
  • Ctrl-C will halt code execution. You can hit enter to go into the REPL. Ctrl-D will soft boot and restart code.py on the device.
  • You can load and save code directly on the device using the editor. Just make sure you also save any changes to your local computer for safe keeping.

Enhancements you might try

• The PyPortal has a touchscreen which we're doing nothing with in this project. What could you do with it?

What you need

• Adafruit Matrix Portal M4
• 64x32 RGB Matrix
• 5V 2.4A or higher USB C Power supply
• USB cable, either USB A to C or USB C to C depending on your computer.
• Free account on Adafruit IO (supports time of day syncing)
• Free account on openweathermap.org (supports weather info)

For a desktop display

• Bent wire stand
• LED Diffusion Acrylic
• Clear Adhesive Squares

How To Do It

Of the three devices, this implementation is definitely the "fancy" version of the three. Along with the information about Instructables Contests scrolling in the bottom line of the display, the top line of the display rotates information about time of day, day of week, date, current local temperature, and current weather condition. A fully featured version that can be the starting point for displaying even more information!

1. As with the other devices, you should learn more about the Matrix Portal M4 and make sure it operates before attempting this project.
   • Review the PyPortal Adafruit Learning Guide to find out how to connect the Matrix Portal to your RGB Matrix, and how to connect it to the Internet.
   • Try the simple project on this page to make sure you have everything working.
2. With your USB cable, connect the Matrix Portal to your computer. It should appear on your computer's desktop as a new drive with the name "CIRCUITPY". (If it doesn't, review the information on this page.)
3. Open two windows on your computer: One to the CIRCUITPY drive and one to the directory where you downloaded the web server project.
4. In the web server project directory, go into the devices folder, and then the MatrixPortalM4 folder.
5. Open the file "secrets_example.py" in a text editor:
   • Update information about your router in the "ssid" and "password" fields.
   • Update information about your web server in the "local_server" field.
   • Update information about your Adafruit IO account in the "aio_username" and "aio_key" fields. You can find this information in your account dashboard by clicking on "My Key".
   • Update information about your openweathermap.org account in the "openweather_loc" and "openweather_token" fields. Details on the openweathermap.org API can be found on this page.
   • Save the file as "secrets.py".
6. Make sure you have the CircuitPython libraries that you need. In the MatrixPortalM4/lib folder, you will find a list of the library directories and files you need in the info.txt file. Information on installing the libraries can be found here.
7. Make sure your web server is running.
8. Copy all the code in the MatrixPortalM4 folder to the CIRCUITPY drive by dragging the files between windows.

After a minute or so, the Matrix Portal should connect to your web server and start displaying all kinds of information including that ever important Instructable Contest data!

BONUS FEATURE!!! The Matrix Portal card has Up and Down buttons on the side, just above the USB connector and just below the reset button. If you click on the the Up button, it will cycle different colors for the top line of the display. If you click on the Down button, it cycles colors for the bottom line. Note these colors only stay in place until the card is power cycled or reset. If you have a particular color combination you like, look at the code to see how you might make that your default.

Desktop Display!!! I'm using a very simple wire stand and acrylic piece to soften the display and stand it upright. You can read more about how to set that up on this web page.

How it works

• There is a lot going on with the Matrix Portal:
  • Time of day information is being synced every hour with the Adafruit IO time service.
  • Weather information is being pulled every ten minutes from openweathermap.org.
  • Contest information is collected when the Matrix Portal first starts, and then a minute after the web server updates.
• There are many CircuitPython options for displaying information on the Matrix Portal. This project uses the displayio module in order to get more control over the display.

Troubleshooting

• As with the previous devices, you can use similar techniques to troubleshoot a device that doesn't seem to be working.

Enhancements you might try

• What data from other public APIs might you want to see if you can mine and display? Adafruit has a ton of examples in their Learning System.

There are a ton of web-based APIs that allow you to get all kinds of information, but as you can see here, web scraping is also another avenue where APIs may not be available. I hope this project has inspired you to explore the possibilities and create a unique project of your own.

Feel free to leave me comments to let me know if you reproduce this project, or modify it to produce something even more useful. Also let me know if you have any suggestions for improvements, either in the projects themselves or the instructions that I'm providing.

I'll update this project from time to time as I make changes and updates.

Happy Programming!