Twitter Watcher, the #twatch

The #twatch scrolls the latest trending topics from Twitter on an LCD screen. It's a stand-alone network appliance that stays updated without a PC. It was awesome to watch #iranelection, Michael Jackson, and other historic events scroll by while we developed the #twatch. This Instructable documents the #twatch hardware and design.

In addition to a Twitter trend ticker, the #twatch is also a generic ethernet LCD backpack. It'll show playlists, PC stats, and other info with programs like LCD Smartie. It's also software upgradable, so it's never outdated.

The #twatch is open source, so you can download our designs and build your own.

Seeed Studio has a few assembled #twatch ethernet LCD packpacks for $45, including worldwide shipping. Get them while they last because we won't make more soon. If you missed this project, sign up here to be notified of future #twatch preorders.

See this article with its original formatting at DangerousPrototypes.com, more discussion in the #twatch forum. We'll send a free #twatch PCB if you're the first to tweet the #twatch!

Concept overview

The #twatch grabs the latest trending topics from Twitter, then loads a few tweets for each. The trending topics and tweets scroll on a screen. The #twatch grabs fresh trends and tweets every five minutes so you always see the latest trending topics.



We also added a generic ethernet backpack mode, so the #twatch can also show PC stats from a program like LCD Smartie, more about this feature in part 2.

The #twatch is easy to use.

• Connect it to a home ethernet network with Internet access. The #twatch requires automatic network configuration (DHCP), this is the default setting on almost every modern home network.
• Power it up. The #twatch requires a 6-7volt DC power supply. It uses a 2.1mm DC power plug, the most common type. Universal DC power supplies should include a 2.1mm plug.
• Adjust the contrast. LCD screens change with temperature and age, use the adjustment screw to tweak the screen contrast.
• The #twatch will configure the network settings and begin scrolling the latest trending topics and a few tweets from each. You may need to adjust the contrast again for the cleanest scrolling effect.

Get #twatch updates at the Dangerous Prototype blog.

We used the freeware version of Cadsoft Eagle to make the circuit and PCB. Download the latest files from the project Google Code page.

This section lost a lot of formatting at Instructables, you can see the original version here.

Ethernet PIC 18F67J60 microcontroller

The Microchip PIC 18F67J60 is perfect for this project because it combines an ethernet network interface and a 41MHz microcontroller (10MIPs) in a small package for only a few dollars. It only comes in 64pin+ TQFP packages, but we didn't have a problem soldering it by hand to a professional PCB.

The PIC requires a 3.3volt power supply. The ethernet portion is really power hungry, so we used a giant TO-220 LD117-3.3volt regulator (VR1). We chose a big regulator because it might have to dissipate a bunch of heat depending on the input power supply. The regulator requires a small input decoupling capacitor (C15) and a large 10uF output capacitor (C3).

There's a big catch with these chips: they can only be programmed about 100 times. That makes development difficult, so we also designed a development version of the #twatch based on a different chip. More about that design in a future article.

Each PIC power pin gets a 0.1uF decoupling capacitor (C17-C23). The PIC has an internal 2.5volt regulator for the microcontroller and ethernet cores, the regulator requires a 10uF tantalum capacitor (C1).

The PIC is programmed through a 5pin ICSP header. The MCLR reset pin is held high with a 10K pull-up resistor (R21), an additional resistor (R4) and capacitor (C16) recommended by the datasheet provide protection against various accidental reset conditions.

The ethernet section mandates a 25MHz external crystal (Q1). Two 33pF capacitors (C4, C5) complete the oscillator circuit.

We used an ethernet jack with integrated magnetics (J2). The jack is a HanRun HR911105A, provided to us by Seeed Studio - be sure to get the same jack, a compatible jack, or adjust the PCB for a jack you can find. The ethernet interface requires a termination circuit (R30-33, C10-11, L1) and 2.28Kohm 1% bias resistor (R7, not shown).

HD44780 character LCD

The #twatch supports a 'standard' 4line by 20character 5volt HD44780 LCD with a +5volt backlight. You can usually find them for about $10 on eBay. Be sure to verify that your LCD matches the #twatch pinout before attaching it. Most LCDs are the same, but not all of them.

Almost all character LCDs operate at 5volts, so we provide a 5volt power supply from a common 7805 regulator (VR2, C14, C2). The LCD with backlight could potentially use a bunch of current, so we used another large To-220 regulator. C12 is a decoupling capacitor for the LCD power supply, but LCDs already have on-board decoupling. C12 need not be populated, we just included it in case of stability issues.

For maximum refresh speed, the LCD is controlled through the full 8bit interface.

Most LCDs are 5volt parts that require about 4.25volts+ to register a high level on the data pins, but the PIC 18F65J60 is only a 3.3volt part. Fortunately, the PIC has a bunch of 5volt tolerant pins so we can hold the signal at 5volts with a 10K pull-up resistor (R10-R19), and then ground it by changing the PIC pin direction setting. This is usually called an open drain output.

Some newer LCDs run at 5volts, but still work at 3.3volt interface levels. The #twatch will support this mode if you leave out R10-19 so no pull-up voltage goes onto the pins, and change the firmware to switch the LAT register instead of the TRIS register in HD44780.c.

LCD screen contrast is controlled by a bias voltage, usually generated with a 10Kohm potentiometer. The #twatch PCB has footprints for an inexpensive 3mm SMD pot (R2), and a second space to use a larger, through-hole pot (R2A). Only one should be populated!

Just in case there's noise in the power supply from all the ethernet stuff, we filter the bias voltage through a small ferrite bead (L2). We also included a capacitor for additional filtering(C13), but we didn't use it since neither element is actually needed.

The #twatch can control simple +5volt backlights up to 400mA or so. The PIC switches a transistor (NPN1) through a 240ohm current limiting resistor (R3, not shown). We used a transistor that can handle 800mA+ with a gain of 250hfe+, so the PIC can switch a large load with it's 20mA maximum pin output current.

R1 is a current limiting resistor for the LCD backlight, if needed. We used a through-hole resistor so it can dissipate a lot of heat with big backlights, and because it's the easiest size to find locally and solder yourself. If your backlight doesn't require a resistor, just replace R1 with a piece of wire. Our LCD required a 3ohm resistor for a 240mA backlight power supply.

Some backlights use a lot of power, so we put the supply pins right next to the power supply and reinforced the ground plane with a bunch of VIAs. Some fancy LCD backlights require special drive circuits, so be sure yours uses a simple +5volt supply to avoid damage.

Power supply

The #twatch requires a 6-7volt power supply through a 2.1mm power supply jack (J1). 2.1mm plugs are the most common size, and should come with every universal power supply.

The higher the supply voltage you use, the more heat that has to be dissipated from VR1 and VR2. Remember that the #twatch is a prototype learning board, it's not a complete and tested commercial product. Take appropriate safety precautions and don't run it unattended.

We used the freeware version of Cadsoft Eagle to make the schematic and PCB. Download the latest files from the project Google Code page.

The PCB is a 2-layer design with small traces and separation (10mil) around the 64pin TQFP PIC chip. We prepared gerbers and sent them to Seeed Studio's PCB service for open source works. Extra PCBs from our order are available in the Seeed Studio shop. If you buy our extra PCBs be sure to get the HanRun ethernet jack that fits the board.

Since the mounting holes on 20x4 LCDs vary, we didn't try to fit the PCB to the LCD holes. We made it as small as possible, like SparkFun's serial LCD backpack, so it stays out of the way of the original holes. As a side effect, it's not too terribly awkward behind some smaller screen like this 16x2 LCD at Adafruit.

Parts list
Click for a full size placement image [PNG].

Part | Value | Package
IC1 PIC 18F67J60 TQFP-64
C1-3 10uF tantalum capacitor, 10volts+ SMC_A
C4,5 33pF capacitor 0805
C10,11,C14-23 0.1uF capacitor 0805
ICSP 5x 0.1" male pin header
J1 2.1mm SMD power jack SMD
J2 HR911105A ethernet jack RJ-45
L1, L2 Ferrite bead, 200ma+ 0805
NPN1 NPN transistor, 250hfe+, 800ma+ SOT-23
Q1 25MHz SMD crystal HC49UP
R2(A) 10K single turn trim resistor 3mm SMD or through hole
R3 240 ohms resistor 0805
R4-6 390 ohms resistor 0805
R7 2,260 ohms resistor, 1% 0805
R10-21 10,000 ohms resistor 0805
R30-33 49.9 ohms resistor, 1% 0805
VR1 LDO 3.3volt regulator (LD1117) TO-220
VR2 7805T 5volt regulator TO-220
HD44780-LCD 20x4 HD44780 character LCD

The latest complete #twatch firmware download is on the project Google Code page. The code is written in C, and is compiled with the Microchip C18 demonstration compiler.

TCP/IP stack and base network functions

Microchip's 'free' TCP/IP stack provides all the network functions we need to exist on a home network and grab data from Twitter. The stack is open source and free-as-in-beer, but the Microchip license prohibits distribution. Due to licensing issues, we're only putting our public domain source code in the project Google Code SVN, learn how to download and compile the source here.

The stack has a Dynamic Host Configuration Protocol client that automatically configures the network settings using the DHCP server on your local network. The #twatch requires a DHCP server, but the vast majority of networks and routers have this enabled. The IP address, mask, gateway, and first DNS server are displayed on the LCD screen until valid Twitter data is available.

The stack also includes Microchip's announce server. When the IP address is acquired by DHCP, the #twatch announces its IP address with a broadcast packet to all computers on the local network. Use the MCHPDetect.exe utility in the project archive to view these packets.

Finally, we included a ping (IMCP) server. Use any ping client to check if the #twatch is alive on the network.

Twitter TCP client

The Twitter trend following program is a simple TCP client, similar to a web browser, that pulls data from web servers. Twitter's API will give us data in a variety of formats. We used the light-weight JSON format because it's easiest for the low-power PIC chip to decode, check out JSONView if you use Firefox.

After the #twatch automatically configures network settings, the Twitter TCP client takes control and grabs the current trending topics. It searches through the this JSON datafeed and looks for the "name" tag. Up to 10 trending topics are copied into a 225byte buffer. A separate array stores the end position of each topic in the buffer so we can retrieve the topics in the next step.

Next, the #twatch searches Twitter for 2 tweets for each topic. It appends each topic to the end of the Twitter JSON search url, special characters like spaces and punctuation are URL encoded.

The TCP client parses through the search results and looks for tweets that follow the "text" tag. Tweets have multiple layers of encoding. We decode HTML reserved characters like ampersand (&) and quotes (") because the LCD screen can display them. We remove UTF8 international characters because the HD44780 LCD doesn't have them in its character set.

The parsed, decoded tweets are stored in a 2100byte buffer, an additional array marks the beginning and end of each tweet in the buffer. RAM space was a big problem on the 18F67J60 chip, it only has about 4000bytes total, but the 2100byte buffer seems big enough to handle 20 average-sized tweets. We took special care to protect against out of memory problems, and we tested the client under reduced RAM conditions to ensure that it fails gracefully during errors.

Twitter is well known for its occasional down-time. If the #twatch can't connect to Twitter, it displays a connection error message and the retries twice. If it can't connect after three tries, it waits five minutes before trying again. This gives Twitter a chance to fix their problems without being hammered by #twatch queries.

The #twatch grabs fresh trend and tweet feeds every five minutes. Twitter puts a limit on the number of queries a client may make, so be careful about refreshing more often. Twitter allows 150 trending topic updates per hour, and "significantly more" search queries.

Network LCD backpack mode TCP server



The #twatch can also display system status information from programs like LCD Smartie. The #twatch has a TCP server on port 1337 that accepts Matrix Orbital formatted commands. This also provides control over the LCD backlight. We'll show you how to redirect LCD Smartie from a COM port to the #twatch TCP server in part two of our #twatch article.

The #twatch can be updated from a PC on the local network thanks to Microchip's internet bootloader. Remember that the 18F ethernet PICs can only be programmed an average of 100 times, so upgrades are somewhat limited. We've yet to burn out a chip, but we only reached about 55 cycles during development.

If you're using a brand new chip you'll need to program the bootloader into the PIC18F67J60 through the ICSP header, then you can upload the #twatch firmware over the network. Program twatchv2-bl-vxx.HEX into the chip with a PIC programmer like an ICD2 or PicKit.

When the #twatch powers on, the bootloader runs before the main program starts. The bootloader checks for a connection between the PGD and PGC pins of the programming header, shown in the image above. If it finds a connection, the bootloader takes over and waits for new firmware to be uploaded.

There's a very minor chance that the bootloader will accidentally start even without a jumper between the PGC and PGD pins. This won't damage the #twatch, just disconnect the power supply and try again. Accidental bootloader entry can be prevented by moving the jumper over one position so it connects the PGD and GND pins.

The #twatch bootloader uses the IP address 192.168.1.123 and subnet mask 255.255.255.0. Your computer must also have an IP address that begins with 192.168.1.xxx to communicate with the #twatch. We chose the 192.168.1.xxx range because it's the most common default for home routers. If your computer uses another IP address range, you'll need to temporarily adjust it before you can do the update.

How to upgrade:

• Ensure your PC is in the same IP range and subnet as the #twatch. Your PC should have an IP address in the range 192.168.1.xxx, and a subnet mask of 255.255.255.0. The default #twatch bootloader IP address is 192.168.1.123, make sure that no other computer connected to the same router already uses this address.
• Unplug the #twatch power supply.
• Put a jumper between the PGC and PGD pins.
• Plug in the network cable, if necessary, and plug in the power supply. The screen may be blank, have solid blocks, or garbage.
• Use a TFTP utility to send the new firmware to the #twatch IP address, we use TFTP.exe from the Windows command line.
• The TFTP update reports success or error.
• Unplug the power supply, remove the update jumper.
• Plug the power supply back in. The #twatch should start scrolling tweets. If the bootloader starts instead, put a jumper between the PGD and GND pins and try again.

We designed the #twatch to make full use of the resources on a single chip, an expanded design would add features but be more expensive.

The #twatch could follow your own Twitter feed. It would need a small web server to enter your Twitter login, and an external EEPROM to store the configuration information.

The #twatch could also store more tweets or extra information about each tweeter, such as name and location. Microchip doesn't make an integrated ethernet controller with more than 4K of RAM, but we could add an external SRAM to store tweets and tweet meta-info.

Updated hardware could add an I/O header for connecting buttons with LCD Smartie.

A 20character by 4line LCD doesn't have a lot of display space. We designed the #twatch interface around this limited amount of space. An updated firmware could handle multiple screen sizes.

The bootloader can adopt the IP address acquired by DHCP. A future #twatch firmware update will take advantage of this feature for easier network upgrades.

Next week we'll cover the LCD Smartie compatible TCP server built into the #twatch.

Get one!
What's twending on your #twatch? If you want an assembled #twatch or PCB, here's a couple options:

• Seeed Studio has a few assembled #twatch ethernet LCD packpacks for $45, including worldwide shipping. Get them while they last because we won't make more soon. If you missed this project, sign up here to be notified of future #twatch preorders.
• If you want to build your own, Seeed Studio is selling the extra #twatch v1 and v2 PCBs from our order. Be sure to get an ethernet jack from Seeed, or make sure you can find one that matches the PCB. We'll write about v1 in a few days, the schematic and PCB are in the project SVN.
• We'll send a free #twatch v2 bare PCB to the first 2 people who tweet the #twatch.

If you'd like to get involved, join the Dangerous Prototypes open hardware project at Google Code, or come chat in the #twatch forum.

Next week we'll show you how to redirect LCD Smartie system stats to the #twatch TCP server.