Bike Embedded Landscape Scanner

Photobike by celian on Sketchfab

In this project, we will be building an embedded addon for your bike, allowing you to scan the landscape you are travelling in wether it is city or countryside.

The goal was to have a very flexible system (where you can tweak parameters for fine tuning your render), able to run hours without problem and without generating too much data.

So I decided to build a system similar to a line scan camera (or basically what does a panorama mode on most smartphones).

The idea is to use a raspberry pi and a camera, to run it on a battery, and to generate single vertical frames based on your speed bike. For that we will need an input trigger on the wheel for a live measurement of the speed.

I also wanted the object to look nice, and to be shock resistant (a race bike having no suspensions, there are some strong vibrations). For these reasons, we will build one side of the frame with carbon fiber (CFRP). This is optional though, in the cad files you will find a variant without carbon fiber.

In order to realize this project and to get the most out of it, it is recommended to have basic knowledge of linux systems and python (you can build it without any programming knowledge, but it wont be as fun).

With a 32 Gb SD card you could have enough space to record for about a 700 km, and with a 20kmAh power bank go for about 20 hours.

The result can look something like this (image with high resolution, here 16.000x1.800 pxl):

The cars being closer to the camera than the 1:1 plan, the appear compressed an look funny

When you find a specific image / segment that you like, you can tweak parameters afterwards to get the best result possible.

• about 300g 3D filament (+ approximately XXg for version without CFRP reeinforcement)
• For CFRP Version: * Epoxy Resin
• 22x12 cm carbon fiber cloth with pattern of your choosing
• A4 or more Plastic film not too loose, and very smooth. I used film for overhead projector.
• 4x M6x25 bolt
• 2x M6x12 bolt
• 6x M6 Nuts
• 5x 5/64 Bolt for fixation of the cover
• 4x 3/32 bolt Raspi
• 4x 3/32 nut Raspi
• 1x M2.5x30 Bolt
• 2x M2.5 nut
• 1x Raspberry pi 3B+
• 1x Raspi Camera v2
• 1x SD card 32g for the Raspi
• 1x Power bank: I chose a 20Ah so i can go for long trips with it. You can go for a smaller one. I t is important to look at the power delivery, it should be at least 2A (the one i chose offers 3A)
• 1x usb cable to connect the output of the power bank (probably USB Type A) to the Pi (micro USB) Length of 20 cm
• 1x usb extension cable >15 cm can come in handy for transfering the videos without opening the enclosure
• 1x Reed switch + Magnet
• some electrical connectors / also female ones for connecting to the raspi pins

• 3D Printer with print surface of 300x300 mm or more
• Display/keyboard/mouse for setting up the pi
• a soldering iron
• Standard tools screwdrivers, allen keys, pliers...
• Multimeter is always usefull with electronics
• a Hair dryer or blow torch can be useful if you go for the CFRP version

You will need to print following parts:

• Cover_Body
• Fixation_Battery
• ixation_Raspi
• ixation_Saddle
• older_Magnet_Sensor
• Holder_Screw_Wheel
• Main_Body(or Main_Body_noCFRP if you are going for a version without carbon fibre)
• Camera_centering
• Camera_Circle
• Camera_Fixation

The files are included in this instructable, but can alternatively be found on Thingiverse.

Most parts are printed with little to no support material with the correct print direction (print direction is also important for the mecanical properties of the parts, so use the same as me):

This will cost you little under 300g of filament, and take about 38 hours in total.

The Carbon fiber and epoxy part can be broken in 4 simple steps:

• prepring the 3D print for casting
• preparing carbon fiber cloth
• casting the first epoxy layer
• implementing the Carbon fiber, and all support elements and seal them with a second epoxy layer

Preparing the 3D Print of the case consists in sealing the bottom face of the print with the plastic film described earlier. To this end i use double sided tape and make sure to leave no gaps and to avoid overlap where resin could flow.

Then you will need to cut your carbon fiber cloth to the correct dimensions. To do this I Stabilised one side of the cloth by puting some transparent tape on the whole surface.

This way you will avoid the cloth falling apart on the edges. You can use a paper guide to cut it with scissors, just make sure to have enough surface to cover all of the visible surface, and 2 or 3 mm margin but not more (otherwise you will end up with Carbon fiber hanging loose in your case, which could be lethal since it does conduct electricity and we plan on putting electronics in there).

At this point you will need to position the part "camera_Circle" and "Camera_centering". The first one is meant to leave a space/hole for the camera to look through. The second one is a guide for making place in the Carbon Fiber cloth. So with double sided tape find a correct position for it (use the camera to see how much space you need).

Once this placed, you can place the guide "camera centering in it, and already make the "hole" in carbon cloth.

Make sure that the cast is set on a stable and very flat surface (for example glass, without dust or anything)

Once this is ready, you can start mixing your epoxy resin according to the instructions on the bottle (mixing ratios may vary so be sure to look for that).

Pour a first Layer of Epoxy, and make sure it is as clean as possible, this will be the visible side. If you want you can use a blow torch or a hair dryer to help evacuate air bubbles trapped in the epoxy. Once your first layer is clean, go on and add the cfrp cloth. try to apply it from one side to the other in order allow for the air to escape. press it gently so that epoxy comes out on the side of it. Then pour a second layer of epoxy on top and guide it on all the sides of the Enclosure. You want to make sure, the bond is as stable as possible.

on top of that second layer, implement the fixations for the pi and camera. Once again, make sure to cover all the sides of those, for a safe bond.

Let it cure for at least 24h then separate it from the support and clean potential small leaks.

Now the main body is ready and ready to receive the components:

Next we will set up the electronic. It is extremely simple.

First you will have to solder cables to the reed switch. Be careful with the sitch, since the main body is made out of glass it is quite fragile. Especially bending the connectors might break it. Go for a cable long enough to go up to your pi.

Once this is done, you can secure the reed switch with a bit of epoxy in the part "holder magnet sensor".

Now you can mount and install the part "holder screw wheel" according to the diagram. Slide a first nut in the slot meant for it in the 3D printed part, scre the second nut all the way on to the screw, then put the screw inside the part up to the first nut. I secured the magnets with electrical tape to avoid losing them in case of a collision.

Now you can mount it on the valve for pumping your bike. You can unscrew the small cap, slide it in, and secure it by screwing the cap back on. It is designed for standard road bike, so 6mm diameter. If your bike is different, you might need to redesign the part.

Now secure the reed switch on the bike at a position where you can reach close to the magnet with some zip-ties.

Now you will need to adjust the position of the screw by screwing it in/out. Once you have found the right position (close enough to the reed sensor without touching it), you can secure this position with the counter nut coming on top of the 3d print (see photos for better understanding).

You now can connect the 2 cables of the reed switch on physical pins 1 and 12 of the raspi. There is no wrong way, the reed switch is just a switch so symmetrical.

You will need to install raspi with standard OS:

1. Download image (do not unzip) from Raspberrypi.org
2. Insert sd card to be flashed into computer
3. Downbload and launch Balena Etcher from balena.io
4. Start belena Etcher
5. Click 'Flash from file'
6. Select your image
7. Click 'Select target'Select your sd card
8. Click 'Flash'

Insert then your SD card into the Raspi.

Connect the raspi to display, mouse and keyboard and boot it.

Connect it to the internet.

From terminal (ctrl+alt+T) run "sudo apt-get update" followed by "sudo apt-get upgrade" to get your raspi up to date.

If you plan on running all scripts on the raspi you will need to get opencv for python: "sudo apt-get install

python3-opencv", but i recommend running it on your desktop computer after transfering the generated video files.

You will now need to download the scripts for image acquisition and image processing from this instructable. Updated versions (if any updates come) can be found here: Link to my Github

Create a folder on the pi where the project will be running where you want and copy the files there.

In order to start automatically the python script on boot i followed the instructable made by scottkildall, which you can quickly go through. The script that must be started on boot is "Autoscript.py", it must be started with python 3.

In Order to access the raspi easily i found very useful to connect to it via SSH. This will allow you to modify files without opening the case once at home.
Depending on the solution you use to connect you can either have graphical interface, or only command line. IN any case you should be used to linux CL and Raspi, if not it will be easier to just connect the pi to display and input devices as usual.

This happens in 3 steps:

• setting up wifi on your Pi: when you have a user interface (raspi connected to monitor)
• connect it to your home wifi
• write down the IP adress of the pi (in linux terminal type "ifconfig",

Then you need to activate SSH on your raspi.

Once this is done you can connect to your pi over network once it is on. For that under windows you can either use OpenSSH Client, use a Linux VM or Linux Subsystem for windows in terminal (my prefered option). If you run a Linux Version ssh is probably already available. User interface will in my knowledge only be possible with either a natively linux system or a VM. IN this case connect with the option "-X" and from there you can start graphical interfaces of programms remotely (like a file explorer)

Connect with "ssh pi@192.168.34.5" for example (replace here the IP you got in the previous step)

Now you can edit the code, remove and or copy files as you wish. As long as your bike is within range of you home wifi, turn on the system and connect to it.

When you are done, you can shut down the pi like this:

You can now assemble all components and mount the system on to your bike.

I recommend using a small USB extension cable for copying the files to a USB stick easily and without opening the case.

Fit the system under your saddle, fix the charged powerbank. Make sure it is in a position that doesn't disturb any movements while on the bike.

You are now ready to go for a ride and scan your city.

Connect the usb power input of the pi to the power bank (and if necessary turn the power bank on). The pi will boot and automatically start the script for capturing videos.

You can now pedal. The more stable you drive the better the pictures.

When you sudently speed up or slow down, the camera will generate
slightly"wrong" pictures, since the framerate only update every 360° of the wheel. Similarly when you turn with your bike, the images will be scanned with the speed of the bike, even though through the turning, the speed of the object you scan is different.

But on straight line, with close to constant distance of scanned object,
and with low vibrations/wobbling of the bike, you can get good images.

Once you captured the videos, copy them on a USB stick to transfer them to your processing computer.

You will then need to convert the videos from format .h264 into something readable by opencv. You can do this with the raspi or on your computer with linux (as before also in windows with windows subsystem for linux). For this install gpac:

sudo apt-get update sudo apt-get install gpac

Then go to the folder with the raw ".h264" videos. Then run following command, which will convert all "h.264" files in the current folder into mp4:

"for i in *.h264; do MP4Box -add $i "${i%.h264}.mp4"; done"

You must run those scripts on a python3 having access to opencv and numpy.

Now we will extract every frame of the ".mp4" videos. In order to do this, we will run the script "Read_video.py". You will need to change the variable "NumberVideosToProcess" to th number of video files you have to process (do not use the last video, since it is not closed correctly by the pi, it is not usable).

The single frames will be saved in a folder "Frames" that you must create before also inside the folder where the videos are.

Now you can run the script "Fusion_images.py".

• You can change the "FusionRatio" (int) which defines the ratio of overlap for a gradient merging of two adjacent images (between 0 and 0.5).

• You can change the "FotoWidth" (int) which is the size of the middle of each frames used for merging (between 1 and 64)

• You can change "NumberFotoPerFrame" (int) whichdefines the number of frames merged into one single image
• You MUST change the variable "NumberImageToProduce" (int) to the biggest (int) smaller than #NumberOfFrames/NumberFotoPerFrame. (NumberOfFrames is not a variable of the programm, you must look how many images are present in the folder Frames.

The image results will be saved in a folder "Result" that you must create before also inside the folder where the videos are.

If you want to tweak the code for varying and playing with the result you can through the parameters explained above. Because of perspective, objects further away move slower, than close objects. Therefore the stretched aspect of the result images will directly correlate with the distance with the object. So if you scan a lon object with constant distance, there is a set of parameter that will give you a 1:1 ratio in X and Y (for example the side of a street). Making scans of objects further away (like landscape) is hard/not possible, since the slightest move of the bike will let the camera aim at a very different spot if it is far away.

This project is made for people who like to do things themselves. The whole process can be further developped for easier user experience.

The System is meant as a Platform for on Bike Embedded System, I implemented a camera because it's a hobby of mine. But the inside Design of the case allows for placement of other components and securing them with epoxy. Anything that you want to do on an embedded system on your bike could be done on this Hardware base.

• Making a mode for standard Video Capture
• Making a mode for Wheel triggered Timelapse (the current code, only image resolution higher)
• Embedded possibility to switch between multiple camera modes
• Logging the Speed of the bike (which is currently calculated with the precision of a wheel rotation)
• Adding sensors (directly controlled by the Raspi, or with a micro controller as an Arduino or Blue Pill for cost efficiency). Sensors such as temperature, humidity, Sound, Vibration... and Logging for somehow implementing it into the Output
• Adding a I²C controlled mini oled Screen (adafruit for example) for status (space on usb Stick, mode selected...) and buttons to control (switch mode, start/stop) directly on the case
• Implementing the camera in order to line scan the ground or the sky. Since woth sky and ground are parallel to the sensor, the unwanted perspective effect on side images would disappear (at the correct frequency)

Overall this has been a fun but laborious projects, with a lot of trial and error, and a lot of potential further functions. I hope to soon be scanning the hole city. If you try to make that project, and encounter some issues, do not hesitate to contact me in the comments.