Evaluation Setup for Real-World Adversarial Patches

This is a guide for building the real-world evaluation setup from the paper Enhancing Real-World Adversarial Patches with 3D Modeling Techniques. Here, we describe both the instructions for modeling the replica of the scene and for building the evaluation setup. Please read and understand all the instructions before you start. The code for the paper is available here. We hope the following guide can be used to replicate our results and support future research. We also encourage future studies to upgrade our setup and supply several suggestions for such improvements.

A special thanks to Mathov Design for helping us to design and build this setup.

Tools:

• 3D printer
• A CNC cutting machine (or carving tools)

Slider:

• A 60x68 cm PVC foam sheet
• 2PCS 8mm linear bearing pillow blocks
• 4PCS 8mm linear rail support
• 2PCS 8mm steel optical axis (or a simple steel rod)
• 2PCS 8mm mounted ball bearing
• Optional: 8mm shaft coupling (depends on your motor)
• 8mm screw rod
• 8mm nut
• 2PCS 20x2.3 PVC foam pieces
• Sandpaper
• Microsoft LifeCam VX-700
• Epoxy glue (or any other strong glue)
• Strong double-sided adaptive tape

Table:

• A wood board: a bit bigger than the PVC sheet
• Legs that will support the wood board: we used leftover pieces and created a table
• 2PCS caster wheels
• 2PCS caster wheels with lock

or an existing table that you don’t need.

We built the scene’s replica according to the attached sketches of the office. The scene is located on the left side, in the central area of the room. The desk is made from two parts, that are connected to create an L-shaped desk. Although the original office also has a small closet and a whiteboard, we didn’t model them as part of the replica. Please note that the sketch was drawn to match the proportions of the real-world objects. The objects are measured in centimeters.

We modeled the replica using Blender and the two sketches. In the Layout workspace, we added the image of the up view of the room (layout > add > image). Then, we added cubes and planes on top of the image and used translation and scale to fit them to the sketch’s measurements with respect to the X and Y axes. Then, we add the image of the side view perpendicular to the up view. Using the new image, we changed the object’s size with respect to the Z axes, which resulted in a 3D replica of the office. This technique was used to create the walls, ceiling, floor, and the two parts of the desk. Later, we added an existing model of the mug (which was created by SEED.EA). To create the patch, we used the Modeling workspace to edit the mug; first, we deleted unneeded faces and then we slightly increased the patch’s scale to fit on top of the mug’s object. Each object was saved as an obj file. The files are available in the author’s repository.

We used a CNC cutting machine to create the grooved base for the slider. Our base is made from a PVC foam board that matches the attached sketch. The CNC can also be used to cut the small pieces for the slider. If a CNC isn’t available, you can carve the groove yourself; if needed, use a softer material (the wood board stabilizes the setup).

It is possible to make the slider and the base wider. However, a long screw rod can band, thus causing the movement of the cart to be shaky and unstable. As a result, the video can be blurry. Therefore, if you need a longer slider, we suggest using a linear guide\rail system instead.

Next, you need to use a 3D printer to print three parts: the cart, inner circle, and outer circle. We supply three files for the inner circle, each from a different height. While we used the shortest part, we wanted to allow other researchers to be able to control the location of the camera. Therefore, if you need to use a higher part, please choose another model.

Use 8 screws to connect the linear bearing blocks to the cart (4 screws each). Glue the 8mm nut to the hole on the bottom of the cart. We recommend checking that the slider works before continuing (as demonstrated in the image): Make sure that the screw rod can be screwed throw the nut for at least 10 centimeters! Next, connect the mounted ball bearing to the edges of the rod and screw them into place. Then, insert the two optical axes throw the bearing blocks, connect the rail support, and screw them in place. Finally, rotate the screw rod in both directions and make sure that the cart slides across the rods.

If everything works, take apart the slider and seal the hole in the bottom of the cart to assure that the nut will stay in place. Then, apply the epoxy glue to the outer circle and fix it on top of the cart. While waiting for the glue to dry, use the inner circle to draw a circle on the sandpaper, and cut it according to the lines. Don’t forget to cut another small hole in the middle of the circle for the screw. Glue the sandpaper to the outer circle, as shown in the picture. By doing so, the inner circle won’t slide when you rotate the camera, and you will have more control over the position of the camera. Screw the inner circle to the outer circle. Finally, screw out and detach the back part of the webcam (marked with a red arrow), and use the double-sided tape to fix the camera on top of the inner circle. As shown in the image, we created another connection point between the webcam and the inner circle by adding a small piece of PVC to the back of the camera.

Once again, make sure that everything works by rebuilding the. This time, you should also check the webcam’s video stream and make sure it isn’t blurry. If it is, consider adding a small piece of PVC between the bottom of the cart and the base. Then, screw the mount and bearing blocks from one edge of the slider to one of the PVC foam pieces (as shown in the image). Make enough place at the edge in the PVC piece for a screw. Consider using a longer piece so you could add a screw in both edges (especially if you are building a longer slider); it will improve the stabilization of the slider. Place the slider on the base (put the fixed PVC piece in the grooved area) and use it to connect the second PVC piece to the other edge of the slider. Finally, check the slider again.

Use any spare materials to build the table for the base. Instead, you can use an existing table (or another piece of furniture) that you don’t need. Make sure that the top of the table is made of stable material, like wood because you are going to screw the base and the slider into this board. The height of the table should be with respect to the location of the scene; the table should be lower than the scene because the camera is located on top of the slider (see our measurements). However, if you use a shorter table you can use a higher inner circle and get similar results.

We built the table from a wood board and spear parts and added four wheels for mobility. If you are also planning to move the setup, consider a way to ensure that you can return it to the exact location. We used two L-shaped steel brackets to connect the table to the same location in front of the scene. Please check the attached images for more information.

Screw the base of the slider to the table.

Configure the rages according to your needs. We defined nine ranges: left, center, and right, for three distances from the scene (i.e., close, middle, and far). For each range, we limited the cart’s horizontal movement using steel eye straps and the edge of the slider (as shown in the picture). We also defined three distances from the scene by moving the slider in the grooved area and fixing it into place using a screw. For each range, we rotated the inner circle and marked its position on the outer circle’s gradations.

The sketch shows the nine ranges (green arrows) that were used for our work. For the three left ranges, the cart moved between the left edge of the slider (measured from the end of the grooved area) and the number of centimeters presented in the sketch. For example, the left range that is the closest to the scene is between the end of the left grooved area and 14 centimeters to the right. The three right ranges are defined similarly. The three center ranges are defined from the point that is marked by the dashed line (this is the distance from the left side of the slider), and its range is defined by the number of centimeters in the arrows. For example, the center range that is the closest to the scene is between 22.5 centimeters right to the end of the left grooved area and 36 centimeters to the right.

Please note that the location of the ranges in the sketch is not in proportion to their distance from the scene in real life. To replicate the distance, fixed the screw on the PVC pieces in the slider (as shown in the picture and sketch) in one of the following locations: 15, 27, or 44 centimeters from the back of the base, for far, middle, and far respectively. The screw on the piece is located 4 centimeters from the back edge of the PVC piece. When you screw in the PVC piece, make sure you screw it to the table. It will make the slider more stable. If you need to improve the stabilization, add another screw to the other edge of the piece.