The Smart-O-Graph

Ever since my schooldays my skills leant more in the direction of science and engineering; I was the best in my class at technical drawing - and the worst in my class at freehand drawing and art. I always wanted to make art, and I think I had the eye for it, but I didn't have the hand for it. If I couldn't use a ruler or a spline to draw with, I was in trouble.

With the marvelous tools we have available nowadays it's possible for us science geeks to turn our hands to artistic matters, as long as we leave the implementation to the computer. I've done a few artworks in recent years where the skill involved was writing computer code, but this week I decided it was finally time to take on art the old fashioned way, and this project is the result...

One way for beginners to learn to draw is to use a device called an Artograph to project an image onto paper which they then sketch over or paint on top of. Actually it turns out that it's not just beginners who do this - recent research has indicated that back in the 1400's the Old Masters started making photo-realistic and accurate perspective paintings using mirrors and lenses. (Some Art Historians still take objection to this suggestion but the evidence is so strong now it's irrefutable by any unbiased observer.)

So I decided to use a device to help me learn to draw, and being a computer programmer I naturally decided to make my device with a computer rather than optically. I demonstrated the resulting device at the McAllen Mini Maker Faire this weekend. Lots of people tried it out with varying levels of success; the best drawing - seen above - was by 5-yr old Joy Hernandez and her mother Juliet working hand-in-hand. Neither mother nor daughter had drawn before. I think it's absolutely wonderful - reminds me of the emotional content of a Mary Cassatt and the economy of line of (dare I say this) a Matisse.

In front of you is a drawing easel, with a projector casting an image on the paper. When you first sit down, the image is a live video window where you position yourself for a head and shoulders photo. The camera is slightly off to one side so that you're not looking straight at the lens. Once you're in position, you click the remote control and it takes your photo. The video image is replaced by the still, and if you're not happy with the picture, pressing the button again will take you back to the video so you can try again. Once you take a photo of yourself that you're happy with, you sit back and wait for about 30 seconds while the computer processes it, and then the image is automatically replaced by a version where an algorithm has drawn over your photograph in a hand-sketched style.

At this point, you can take a pen and draw over the lines yourself on paper. With very little practice you can produce an amazingly good drawing - all the drawings shown here were done the very first time the drawer used this system.

I drew these images of my wife and myself. The one of myself was literally the first drawing I made with the system. My normal standard of drawing is a stick figure. Seeing this emerge was just unbelievable. The next one I did was my wife. I switched from Biro to using a thick Sharpie, and I think the resulting look is a little more cartoonish but still extremely recognisable.

These two ladies had drawn before, but the speed at which they knocked these out was amazing.

The software that adds the sketch to the photograph uses an NPR (Non-photorealistic rendering) algorithm written by university researchers from the University of Missouri, St Louis (http://www.cs.umsl.edu/~kang/Papers/kang_npar07.html).

I wrote a C version of the algorithm some years ago for my Scratchboard etching project, and used the ImageMagick package to superimpose the line drawing over the photograph.

Almost any computer will do, as long as it has a webcam that can be used to capture video and stills. I used a Raspberry Pi and a Raspicam. For the display I used a pocket laser projector. A laser has the advantage of always being in focus, although the projector I own was not ideal as it used a composite video signal rather than the Pi's preferred HDMI. However resolution isn't an issue for this project as no text is ever displayed. You could also use a full sized projector if you could work out how to position it to cover the paper without being blocked by the user. (A mirror might be helpful there)

I mounted the projector on an extensible stand that was originally intended for hanging a Dremel from. To give the projector a stable base to attach to, I pushed the pole into a block of packing foam. Ugly, but worked pretty well.

The Raspberry Pi (with camera built in to a case) was attached to an easel using a piece of curved plumbing material I had to hand; the curve allowed me to position the Pi at the correct angle to face the drawer. Yes, it's stuck on with double-sided tape and rubber bands. Despite that it stayed on solidly and worked well.

I deliberately avoided using a keyboard and screen once I had the software working. To use a remote control, I attached a FLIR USB remote receiver, and coded a single button on the remote to send a control-C character when pressed. This allowed me to forcibly kill any processes that were running and move on to the next step without worrying about what key was expected, regardless of what was running at any time the user might have pressed the remote. Obviously the bash script that was controlling the system had to be modified to take this into account by trapping interrupts and handling them appropriately.

(the image above shows a projector screen behind where the drawer sits. It's only there to provide a plain background for the initial photo; it's not used by the drawing system. Although for the Maker Faire I did set up a spare video camera pointing at what the user was drawing so that the crowds who came to watch could all see without crowding around the person in the chair)

When I put this together over a weekend for fun (and because I'd been asked to exhibit at our local Mini Maker Faire), I didn't have any intentions of taking it farther. However, there was a lot of interest at the Faire and a couple of art teachers from local schools said they'd be interested in using this in their lessons. I'm not planning to build more but I might keep this one assembled for a few weeks to let those schools borrow it.

I called this a "Smartograph" as a pun on the commercial "Artograph" product which is a non-computer optical device; I saw this as a smarter version of the Artograph. If the Artograph people find this post and are interested in developing a computerised tool like this, I'm making no claims to the intellectual property and they're welcome to take the idea and run with it - although I wouldn't say no to being hired as a consultant on a commercial project like that :-)

What's next for me personally in creating art? My wife suggested that we try a painting project to see how easy it is to replicate the style of the Dutch and Italian old masters. I'll post here if we do it.