Arduino Development Tool

What great fun the Arduino is but...

The IDE serial monitor just isn’t sophisticated enough for my liking. I found it difficult to get a mass of information displayed meaningfully. The monitor whizzed past and intermixing different types of information just made the display a muddle of text.

I tried Processing for outputting my IMU which was very informative but was prettier than I needed and I found myself struggling to add more output information especially when I wanted it to be running concurrently.

So I wrote my own serial monitor and if there is enough interest I will also look into making it available.
I can’t really show you all the code as there is masses of it but I will provide some hints and tips for anybody who wants to have a go themselves.

It’s been tested on an Arduino UNO, Mega256, Nano and Pro Mini (all clones) and also on self builds using the ATMega328 with a CP2102 for the USB serial connection.

This was developed under .NET for the Windows platform and has the following forms/visualisers:

• Main monitor
• Message sender
• Trace copy
• LCD emulator
• Freeform drawing surface
• Alert reporter with  ability to run external programs
• PS3 controller visualiser
• XY scatter plotter
• Sweep plotter
• Histogram plotter
• IMU visualiser

These can be controlled and configured from the PC and/or from the Arduino. To make the above forms respond to the data stream a simple protocol is used with commands identified with a # prefix and parameters separated by a ~ character. Free text is supported for viewing in the main monitor. All window forms have a quick button back to the main monitor window, a clear/reset button, a pause option, tracer text for that particular window form and a test button to try out commands directly as if sent via the serial port.

I adopted text representations rather than binary so it is possible to ‘see’ what is going on in the Arduino IDE’s serial monitor which aids development speed. It may not be as compact or fast as binary (which could be made more data efficient) but its convenience outweighs this benefit and the typical PC is well able to cope - it is possible to have all of the above forms being updated simultaneously from an Arduino running 19200 baud.

This can be used for other serial sources than the Arduino should it be needed to.

Planned enhancements include:-

• A scripting tool so that messages can be sent to the Arduino based on the content of a script file stored on the PC.
• User defined visualisers. These will be constructed from definition files that allow custom forms to be built up from standard components such as buttons, sliders, graphing tools so it can receive and send data to the Arduino.
• Data out formatted for use in third party systems include Microsoft Excel, XML, CSV and so on.
• Custom visualiser for tuning common subsystems such as PID (proportional, integral and derivative) controllers and IMU filtering.

Main monitor window 
 

• This allows selecting, refreshing, opening and closing the COM port.
• Choosing a reading strategy - fast read normally and one line at a time where more control over the sequencing of messages to and from is important.
• Control the display - pausing, clearing and copying the trace, adding in a timestamp which also shows whether the data flow is inbound, outbound or looped back (for test purposes where the serial device does not need to be connected).
• Choosing a display refresh rate to optimise the loading on the PC when large quantities of data are being transmitted. Whilst reading the trace at high speed is difficult it is useful for the real time visualisers such as the IMU.
• Read and display buffer usage is displayed to allow tuning and warn of either the PC or the Arduino generating too much data for the other.

 

 Message Sender 
 

• This allows data to be sent outbound on the COM port and controls whether the output is included in the trace window.
• Control whether the output is not sent out but simply looped back. This is used for convenient experiments with data streams to see how they work with the various forms and visualisers.
• A looping tool allows multiple parameter substitution and repeats.
• A joypad simulation with user adjustable data.
• A keypad simulation with user adjustable data.
• A slider tool with user adjustable data.

Trace copy 
 

• This allows you to review the trace output while the main trace window is running.
• This contains the text of the trace window when requested for by either the user or the Arduino.

LCD emulator 
 

• 80 columns by 25 rows.
• Write text anywhere on the ‘screen’ in any of 7 colours for foreground and background.
• Scroll command to you can treat it like a TTY display.
• Make up your own user interface.

Freeform drawing surface 
 

• 500 pixels by 500 pixels drawing surface.
• Commands for line, rectangle ellipse, arc and pie, filled and unfilled, cross and text, in various sizes and orientations and in 5 colours.
• For example plot out your robot path.

Alert reporter  
 

• Somewhere to report special events to the user.
• Option to send run commands to start programs.
• Beeps to alert user

PS3 controller visualiser 
 

• Displays PS3 or similar controller button and joystick values.
• Green shows values unchanged since last reading.
• Orange shows values that have changed since last reading.
• Red shows values that are at maximum.
• Joystick position cross hairs.
• Useful for watching what the controller is sending to your robot for example.
• See how sensitive the different buttons are.
• Note if the joysticks are not self centering or minimum/maximum values not reached.
• Pause ‘able. 

XY scatter plotter 
 

• See the relationship between two values for up to five different sets.
• Easy user control of axis settings.
• Pause ‘able. 

Sweep plotter 
 

• Trace like oscilloscope.
• Five traces with a resolution between 10 and 1024 points.
• Easy user control of axis settings.
• For example watch accelerometer and gyro readings.
• Pause ‘able.
• Auto scaling option.
• Toggle between data and keys.
• Pause ‘able. 

Histogram plotter
 

• 21 bar histogram.
• Shows percentage of observations and count of observations.
• Easy user control of axis settings.
• Over and under range outlier recording.
• Simple statistics - mean, min and max of within range observations.
• Pause ‘able. 

IMU visualiser 
 

• Roll and pitch visualisation.
• Invert option to suit installed orientation of breakout board.
• Accelerometer and gyroscope readings (including gyro drift summation).
• Pause ‘able. 

Hints for those wanting to do it themselves
 

• It was interesting to discover that to my UNO and Mega did not work reliably at 300 baud and the buffering of data between them and the PC is also different. These differences are not noticeable in ‘normal’ conditions but show up at the limit of performance such as when transmitting a lot of data very quickly, in either direction and at the lower or higher baud rates. Make sure you test at the full range of performance and with different hardware as they do not behave exactly the same.
• .NET exception handling isn’t very fast so do all your own data checks otherwise the program will slow and the Arduino will send out data faster than it is being processed on the PC.
• Multi threading is a must as you need to get the reading of the serial input stream done as fast as possible and into buffers the various forms can the handle for displaying to the user.
• Data can come so fast that it is impossible for a user to read every change unless it is a plot type of display. Therefore don’t bother to update display data for every single character as it changes. This will clog the PC down even though the display processing is done in a separate thread - remember there is only a finite amount of processing available and multi threading doesn’t make more of it, it just reassigns it differently.
• Factors outside of the Arduino and the serial monitor can affect its performance. I leave the program on soak testing overnight receiving data pumped out at maximum speed from the Arduino. That means thousands and thousands of lines of data and I’ll notice that at say 2am in the morning one line will be in error - typically the length it is and the length it should be don’t agree. I can only assume a virus check has kicked in and messed something up. While this may be an exceptionally small percentage of the data it might result in a serious consequence in your system.
• Check the data received is expected and error handle if it is not. For all the special commands used in the serial monitor I send the length of the data as well as the data from the Arduino and I also check it on the serial monitor. If they don’t agree I don’t send the data to the appropriate visualise form but send what has been captured to the Alert visualise for the user to aware that something has gone wrong. I have considered using a CRC checksum but the added processing time doesn’t, for me, warrant it over simple line length checks.
• As much as you would like to avoid getting the user involved in tuning the system at the extremes of what your PC can cope with it can buffer up a lot of data which means the visualisation displays lag behind the Arduino’s actual behaviour. In my first versions of this tool I surfaced a lot of tuning parameters before settling on two that were really useful/unavoidable. Reading all bytes as soon as theyare available versus the much slower read a line at a time (useful when you are sending data back to the Arduino and scheduling it into the correct sequence is important) and slowing the displays down to allow them to update in ‘chunks’ makes it easier to read fast changing listings.
• Don’t expect it to be right first time. I had a couple of technical proof of concepts (during which I found that some code available on the Internet was faulty with a slow but steady memory leak). Once it works fast and reliably then you can refine the user interface to give the features you need and also make them convenient to use. This is version 3.
• Processing binary data may be more compact/faster but cannot be viewed in the Arduino IDE which is a convenient fall back option to testing.