Seeing Sound Waves: Bug-Sweeping Robots for Augmented Reality

(top picture) microphone hidden on bookshelf;

(bottom picture) another microphone hidden inside this cute stuffed animal's nose....

This Instructable introduces "ARbotics" (Augmented Reality robotics), and in particular, robotics for phenomenological augmented reality.

Let's begin with a simple example: seeing otherwise invisible sound waves, as well as metasensing (metaveillance) of sound.

What is metasensing (metaveillance)?

A meta-conversation is a conversation about conversations. A meta-joke is a joke about jokes. Metadata is data about data (like the GPS coordinates that record where a picture was taken, which are embedded in the JPEG header information of the picture).

Likewise, metaveillance is the seeing (veillance) of sight or other senses. Metasensing is the sensing of sensing, or the sensing of sensors, or sensing their capacity to sense.

A nice simple example to begin with is the sensing of microphones, and the sensing of their capacity to sense. In order to make this sensory data visible, we use the S.W.I.M. (Sequential Wave Imprinting Machine), invented by S. Mann in 1974, which was the subject of a previous Instructable.

The SWIM may be mounted to a robot. Various kinds of robots can be used. Perhaps the simplest approach is to make a slide rail, rail car, or the like, or use a motorized toy car or truck, or both (e.g. mount a simple reciprocating robot in the back of a radio-controlled pickup truck). A large pickup truck is a good choice (as I used in the above figure) because it is easy to mount a motorized sliderail and rail car onto the open cargo area of the pickup truck, which can swing the SWIM back and forth while the truck can move forward through the space, thus automating the process of bugsweeping.

I built a working prototype for this project a long time ago, using a Tamiya TAM58372 Tamiya 58372 Ford F350 High-Lift Truck Kit Model Kit, but that same kit happens to still be available from Amazon.com, or you can usually get an old radio control model kit, car, truck, boat, drone, or the like from garage sale, flea market, or dumpster (people often throw away perfectly good radio control toys just because of some very minor problem you can very easily fix).

The "Bugbot" is not merely a robotic bug sweeper, but, more importantly, it is an augmented reality visualization machine.

A little bit of machine learning can go a long way here, and the simple "Bugbots" you build can form the basis for a wonderfully fun research program in ARbugbotics.

ARbug-bots are great for teaching, too. What's there to not love about using toy cars to find hidden microphones in cute little stuffed animals? Awaken the inner-child in your mind, with this spy-versus counter-spy: spybot versus counterspybot narrative: a great way to teach robotics and science at the same time!

So let's get started! We'll begin by measuring the speed-of-sound (or light) and making it "sit still". This will give us "sitting waves" that make visible the capacity of a microphone to hear.

The first thing to do is make a SWIM (Sequential Wave Imprinting Machine) as shown in one of my previous Instructables.

The SWIM will form the basis for unlocking the power of AR to make visible the otherwise invisible sound waves, radio waves, gravitational waves, and many other waves that surround us.

Mount the SWIM on a rail or structure that allows it to move back-and-forth, and then you can motorize this back-and-forth sweeping movement -- -- RoboSWIM!

Paint everything black if you can, so that only the lights are contributing to the visuals.

The secret to success is a good lock-in amplifier.

Bug sweepers usually work by making some kind of sound in the room and then sensing feedback, e.g. that screech or squeal that happens as a result. Most bug sweepers are not very sensitive.

Lock-in amplifiers are good at being sensitive, and a good amplifier has a gain of about a billion times, e.g. from 1 nanovolt full scale to 10 volts full scale is a gain of about 10 billion.

A good amplifier also has a good "dynamic reserve" on the order of 100dB or so or maybe 120dB, meaning that the signal you're trying to pick up can be being drowned out by noise that's maybe a thousand times stronger than the signal you're interested in picking up.

The best lock-in amplifier ever made is the PAR124A made by Princeton Applied Research in 1961.

That was 55 years ago.

Nothing made today comes even close to it.

If you can find one, treasure it, handle it lovingly, and take good care of it!

Jim Tucker used to sell old refurbished PAR124A amplifiers four around $6000 US, not including the preamplifier (the transformer alone in the PAR116 preamp is worth about $4000 today).

I bought a new top-of-the-line SR865 lock-in amplifier (about $12,000) and it was useless for bug-sweeping, so I sent it back!

There's another product made by Signal Recovery that claims to do as good as the 124A but its also no good (at about $17,000 ... I also sent that one back too).

Pictured above:

1. PAR124/126 hybrid (with Mann modifications customized for bugsweeping): best.
2. SR510 adequate
3. SR865 not good (in my opinion)
4. SR7280 no good (in my opinion)

Advertisement from a physics magazine also shown. The new one's (in my opinion) no good, though.

In a future Instructable I'll post how to make a low-cost lock-in amplifier that outperforms even the $60,000 unit from Zurich Instruments AG (for which you'll probably need to do a public tender if you're buying through company or gov't grant).

In the meantime, as a general rule of thumb, if it was made after the 1960s or 1970s, its not much good!

Note to self: write the Instructable on the ultraminiature WearTech.com wearable lock-in amplifier that outperforms all of these....

The way the SWIM works is that you connect a stationary sensor/effector (transducer or antenna) to the reference input of the LIA (Lock-In Amplifier), "R", in the above diagram, and the moving sensor/effector (transducer or antenna) to the signal input of the LIA ("S" in the above diagram). The in-phase output of the LIA ("X" in the above diagram) is connected to the SWIM.

The reference sits near where you think there might be a bug, and the moving sensor/effector is attached to the SWIM which is the "sweeper" moving back-and-forth.

Experiment with different configurations. You can experiment with a stationary source of excitation (e.g. a speaker) and a moving microphone, and vice-versa, until you get things just right and are able to "lock on" to a very weak signal that's buried in noise.

Tinquiry (tinkering as a form of inquiry) is the key here: keep experimenting until you get something you like!

Let's begin with a rail and railcar, and a simple experiment in which you'll measure the speed of sound (or light or other...) and, more importantly, put yourself in a frame-of-reference that effectively moves at the speed of sound (or light, or ...) and thus makes the sound (or other) waves appear to "sit" still.

In the above picture, we're teaching a class on phenomenal augmented reality (ECE516) and each student or visiting student is invited to slide the railcar back and forth and see a sound wave as if it were sitting still.

Here, for simplicity, we have a microphone transmitting to a receiver, differential input, to a PAR124/126 lock-in amplifier (Princeton Applied Research Lock-in Analyzer), which effectively performs the operations that are mathematically depicted on the chalkboard drawing. You can also build our own amplifier, although the PAR124 is quite possibly the best amplifier ever made (until recently its 2.5 nanovoltseconds noise figure remained untouched, except now by the SR124 or SR865 manufactured by Stanford Research Systems). It is easy to build an amplifier that will show the concept, but hard to beat the legendary performance of the PAR124 in terms of sensitivity.

The output of the amplifier goes to the SWIM, essentially showing therefore the homodyne voltage of the mixer with a reference signal selected and set to, in this case, the frequency was dialed in around 10,000 CPS (Cycles Per Second) and then verified with a digital frequency counter. The exact frequency was 10.05kCPS (10050 CPS).

The run length along the rail was 0 to 15 inches. The set screw knob pictured here protrudes in front of the railcar, so the perspective of the image makes it appear to the left at left, and the right at right, but the actual distance is right at base close to 0 and 15 inches, i.e. 15 inches total travel.

We can see that there are 11 cycles of the sound wave here.

15 inches * 10050 cycles per second / 11 cycles = 13704.5454... inches per second = 348.09... meters per second.

The speed of sound at 20deg. C is about 343.2 metres per second, but our lab was about 25deg. C, so the speed of sound in our lab was about 343.2 +.606 m/s/deg *5 degrees difference = 346.23m/s.

So we have about 1/2 a percent measurement error.

For teaching purposes (to teach yourself or others), you can also try a frequency of 13631.0751 CPS, which would be 1 inch per cycle at 25deg. C., but some microphones, including this one don't work as well at that high a frequency.

Here's a robot Kyle and I built to wave my SWIM back and forth.

You can see it sitting on a workbench we built from sound waves swept out in this manner (if you like the table/workbench that's made of sound waves, here's how you can make one yourself).

Here we're visualizing electromagnetic radio waves from a microwave motion sensor.

You can sweep for sound, radio, or whatever else you like!

Now throw it in the back of a self-driving pickup truck, so you can go forth and sweep the world.

(Long-exposure photographs of a SWIM attached to a reciprocating robot sitting in the cab of

If you're new to all this, here's a fun challenge that's not too difficult.

Simply begin by building some kind of electromechanical apparatus that waves a stick back and forth quickly and smoothly and safely and reliably.

If you get that far, please click "I made it" and post your results.

Even if you don't get the bug sweeper working, just put some fixed LEDs on your stick and take a few long exposure photographs and videos of it swinging back-and-forth, and post these as "I made it" to give me an idea of what you can do.

I'm not very good at robotics, so I'd like to collaborate with someone who is.

If you're good at robotics, post some pictures and if you happen to live in, or are visiting the Bay Area ("Silicon Valley"), I'd like to meetup with you and potentially collaborate on making some great pictures (e.g. attaching one of my SWIM devices to your robot, connecting it to my best-in-the-world lock-in-amplifier array, and making some really great artwork together).

I want to see what different people come up with for something simple that moves back and forth.

For my students, this is your first assignment in the world of making things.