Braille Vision: a Portable Text-to-braille Device

78 Views, 2 Favorites, 0 Comments

Braille Vision: a Portable Text-to-braille Device

The aim of our project, Braille Vision, was to improve media accessibility for the visually impaired, by letting them read text around them that has no braille transcriptions available. It works by first taking a photo of media such as a poster that is inaccessible to the visually impaired, reading the text from it, before converting this data into a VI accessible format (braille) via a braille display pad. Braille Vision performs this text-to-braille task through use of a Raspberry Pi that uses tesseract OCR to read text and send the data to an Arduino, which then controls the braille display pad.

We think that this project is a great way to start learning about the vast range of practical applications you can achieve with a Raspberry Pi, (this being our first project using one) as well as more general electronics, arduino and braille.

here's a demonstration of the finished product

Supplies

-Raspberry Pi 4 or 5 (At least 8gb recommended)

-if you haven't set up a Raspberry Pi before, you'll also need a high speed SD card (we used the SanDisk 64GB Extreme PRO), a micro HDMI to HDMI cable and suitable power adapter. For a full guide on setting up for the first time, I recommend Tom's Hardware.

-Raspberry Pi Camera Module 2

-Raspberry Pi Case (We used this one)

-Arduino Uno

-Soldering setup

-3d printer

-Hot glue gun

-1 piezo transducer

-1 rotary encoder

-3 10K resistors

-6 IRLZ34N MOSFET transistors

-6 1MΩ resistors

-6 1n4007 diodes

-6 mini solenoids (I used the pimoroni COM2700 https://shop.pimoroni.com/products/mini-solenoid?variant=2284520931338)

-A microswitch (We used this one)

-Some perfboard

-Lots of wires

-7.4V LiPo battery

-A 5v portable charger

-6 3M heat set inserts

-8 3M bolts

Learning Braille

WhatsApp Image 2025-04-24 at 23.13.28_90aa8f7a.jpg

In order to design a product that converts text to braille, we first had to learn braille. I highly recommend printing out the braille alphabet using a 3d printer and practicing with the physical thing. There are many open source 3d models available online - I sourced mine from Maker World, as I use a Bambu labs printer.

We also found the standard dimensions for braille published by the UKAAF, as we had to follow these for the design of our braille display pad, ensuring it has the same proportions and feels almost the same as any other piece of braille printed in the UK.

Set Up the Raspberry Pi

We first have to set up the Raspberry Pi with a monitor. We did this using an online guide (Tom's Hardware). After setting it up, you can attach the camera and test it by running the command

libcamera-still -t 5000

in a terminal.

Once this is done, you can install the necessary packages.

sudo apt update && sudo apt -y upgrade

# Install tesseract

sudo apt install tesseract-ocr libtesseract-dev libleptonica-dev pkg-config

``` :contentReference[oaicite:4]{index=4}

# Verify Installation

```bash

tesseract --version

# Create virtual environment

sudo apt install python3-venv

python3 -m venv myenv

source myenv/bin/activate

# Install Python packages

pip install opencv-python pytesseract pyserial

pip install pillow numpy opencv-contrib-python

After all the packages are successfully installed, we can write the code in the text editor. In an editor such as Thonny, you can copy and paste the following code and save the file as "Braille.py".

#!/home/PotatoPi/myenv/bin/python

import time

import subprocess

import cv2

import pytesseract

import serial

# Set up Serial communication with Arduino

ser = serial.Serial('/dev/ttyACM0', 9600) # Update with your correct serial port

time.sleep(2) # Allow time for Arduino to reset

# Step 1: Wait for signal from Arduino to take a photo

print("Waiting for signal from Arduino...")

while True:

if ser.in_waiting > 0: # If data is available from Arduino

signal = ser.readline().decode().strip()

if signal == "START": # When Arduino sends "START", proceed

print("Signal received, taking photo...")

# Step 2: Capture image with libcamera

image_path = "/home/PotatoPi/abn.jpg"

subprocess.run(["libcamera-jpeg", "-o", image_path, "--width", "1280", "--height", "720"])

print("Photo captured: ", image_path)

# Step 3: Process the image for OCR

image = cv2.imread(image_path, cv2.IMREAD_GRAYSCALE)

gray = cv2.threshold(image, 128, 255, cv2.THRESH_BINARY)[1]

# Extract text from image

text = pytesseract.image_to_string(gray)

print(f"Extracted Text: {text}")

# Step 4: Send text to Arduino

ser.write(text.encode())

print("Text sent to Arduino.")

Now, we want the code to run on the Raspberry Pi without having to use a monitor, keyboard and mouse. In order to do this, we can make the python script run on boot. To do this, first execute this in terminal

chmod +x /home/PotatoPi/myscript.py

open the crontab file by writing:

crontab -e

At the bottom of the file, add this extra line of code:

@reboot /home/pi/Braille.py &

before pressing Ctrl + C, Y, Enter.

Early Iterations

This project took many hours of brainstorming, sketching, 3d modelling and iterating. I've attached some sketches and printed models to give you an idea of the way this design has evolved over the course of many iterations.

Final Design Modelling and Printing

After completing the final design on Fusion 360, we 3d printed all the pieces. For the solenoid to braille dot adapter pieces, a .2mm nozzle makes a huge difference in quality and I would advise the use of one if possible. If all you have is a .4mm nozzle, you may need to sand down the adapter pieces afterwards.

I have attached all the necessary 3d models below in STEP format.

Downloads

Braille converter v11.step

Short braille conversion rod v4.step

long braille conversion rod v2.step

Solenoid array v7.step

Pin covers v4.step

Braille-side panel v19.step

camera-side case v23.step

ms pins v3.step

cam_pin v3.step

knob v3.step

Electronics

We first planned out the electronics by drawing some sketches and finding diagrams online. The circuit design we created uses MOSFETs as drivers so that the low current output of the Arduino can control the high current draw solenoids. If you want to better understand how these transistors work, I recommend watching this youtube video by the engineering mindset.

Soldering and Assembly

After planning all the electronics out, we soldered everything together. You can do this by referencing the sketch of the solenoid driver circuit and rotary encoder diagram in the previous step. We produced 3 pairs of mosfet driver circuits on 3 pieces of perfboard and left everything else (the rotary encoder, microswitch and piezo transducer) without perfboard - directly soldered onto wires. We then attached these to the Arduino in the following arrangement:

microswitch pin - 10
rotary encoder pin A - 2
rotary encoder pin B - 3
rotary encoder switch pin - 12
piezo transducer - A0

We then secured the loose pieces in the plastic casing with the help of some hot glue. The positions of all the pieces can be found by looking at the 3d model of the assembly which I have attached to this step.

Downloads

Text-To-Braille Assembly rev2 v15.step

Arduino Code

Download the following code onto the Arduino Uno.

// Function Prototypes

void PinA();

void PinB();

void recvWithEndMarker();

void showNewData();

void printBraille();

int ASCIItoInt(char value1);

// Global Variables

int msPin = 10; //the microswitch pin

int buzzer = A0;

static int pinA = 2; // Our first hardware interrupt pin is digital pin 2

static int pinB = 3; // Our second hardware interrupt pin is digital pin 3

volatile byte aFlag = 0; // lets us know when we're expecting a rising edge on pinA to signal that the encoder has arrived at a detent

volatile byte bFlag = 0; // lets us know when we're expecting a rising edge on pinB to signal that the encoder has arrived at a detent (opposite direction to when aFlag is set)

volatile byte encoderPos = 0; //this variable stores our current value of encoder position. Change to int or uin16_t instead of byte if you want to record a larger range than 0-255

volatile byte oldEncPos = 0; //stores the last encoder position value so we can compare to the current reading and see if it has changed (so we know when to print to the serial monitor)

volatile byte reading = 0; //somewhere to store the direct values we read from our interrupt pins before checking to see if we have moved a whole detent

int pinSW = 12; //switch (pressed Encoder)

int state = 0;

bool letterState; //High if letter, LOW if number

const int MAX_TEXT_LENGTH = 512; // Define max text length that can be received

char receivedText[MAX_TEXT_LENGTH];

char receivedChars[25]; //An array of size 25 is used to store the incoming data.

int numChars = 25;

int charArraySize;

int arrayNumber;

int tempCharNum;

boolean newData = false;

int mosfets[] = {4,5,6,7,8,9};

bool printReadyState = false; //this tells the code when the array is full and ready to print.

bool lastNumberState = false; //tells you if the last character was a number

bool capitalLetter=false;

bool number=false;

bool punctuation=false;

bool letter=false;

int alphabet[27][6] = {

{0,0,0,0,0,0},

{0,1,1,1,1,1},

{0,0,1,1,1,1},

{0,1,1,0,1,1},

{0,1,1,0,0,1},

{0,1,1,1,0,1},

{0,0,1,0,1,1},

{0,0,1,0,0,1},

{0,0,1,1,0,1},

{1,0,1,0,1,1},

{1,0,1,0,0,1},

{0,1,0,1,1,1},

{0,0,0,1,1,1},

{0,0,0,0,1,1},

{0,1,0,0,0,1},

{0,1,0,1,0,1},

{0,0,0,0,1,1},

{0,0,0,0,0,1},

{0,0,0,1,0,1},

{1,0,0,0,0,1},

{1,0,0,0,1,1},

{0,1,0,1,1,0},

{0,0,0,1,1,0},

{1,0,1,0,0,0},

{0,1,0,0,1,0},

{0,1,0,0,0,0},

{0,1,0,1,0,0}};

int punctuationList[16][6] = {

{0,0,0,0,0,0},//space

{1,0,0,1,0,1},//exclamation mark

{1,0,0,1,1,0},//quotation marks

{1,1,0,0,0,0},//hash - number sign

{1,0,1,1,0,0},//dollar sign

{0,0,0,0,0,0},//percentage sign. Requires 2 characters

{0,0,0,0,1,0},//and

{1,1,0,1,1,1},//apostrophe

{0,0,1,1,1,0},//open bracket (2C needed)

{1,1,0,0,0,1},//closing bracket (2C needed)

{1,1,0,1,0,1},//asterisck (2c needed)

{1,0,0,1,0,1},//plus (2c needed)

{1,0,1,1,1,1},//comma

{1,1,0,1,1,0}, //dash (2c needed)

{1,0,1,1,0,0},//full stop

{1,1,0,0,1,1}};//forwards slash (2c needed)

int numberSymbol[6] = {1,1,0,0,0,0};

int capitalSymbol[6] = {1,1,1,1,1,0};

int letterSymbol[6] = {};

int ASCIItoInt(char value1){

int charVal;

int temp = value1;

bool capitalLetter=false;

bool number=false;

bool punctuation=false;

bool letter=false;

if(temp>96 && temp<123){

charVal=temp-96; //this way, a is number 1, not number 0 - 0 is reserved for the full character (all solenoids off)

letter=true;

}else if(temp==32){

charVal=0;

punctuation=true;

}else if(temp>64 && temp<91){

charVal=temp-64; //again, we set A to 1, as number 0 is reserved for {0,0,0,0,0,0}

capitalLetter=true;

}else{charVal = 0;

punctuation = true;

}

return charVal;

}

void setup() {

for(int temp=0; temp<6; temp++){

int var = mosfets[temp]; //set all the mosfets to outputs

pinMode(var, OUTPUT);

}

pinMode(msPin, INPUT_PULLUP);

pinMode(buzzer, OUTPUT);

pinMode(pinA, INPUT_PULLUP); // set pinA as an input, pulled HIGH to the logic voltage (5V or 3.3V for most cases)

pinMode(pinB, INPUT_PULLUP); // set pinB as an input, pulled HIGH to the logic voltage (5V or 3.3V for most cases)

attachInterrupt(0,PinA,RISING); // set an interrupt on PinA, looking for a rising edge signal and executing the "PinA" Interrupt Service Routine (below)

attachInterrupt(1,PinB,RISING); // set an interrupt on PinB, looking for a rising edge signal and executing the "PinB" Interrupt Service Routine (below)

Serial.begin(9600);

while(!Serial){

; //wait for connection with the pi

}

void loop() {

// Wait for button press (goes LOW)

while(digitalRead(msPin) == HIGH) {

delay(5);

Serial.println(digitalRead(msPin));

}

// Button pressed; now wait for release (goes HIGH again)

while(digitalRead(msPin) == LOW) {

delay(5);

}

delay(150); // Debounce delay

tone(buzzer, 440, 250);

Serial.println("START");

while (Serial.available() == 0) {

// wait until data comes in

}

recvWithEndMarker(); // Receive characters until '\n' is encountered

tone(buzzer, 1200, 1500);

if (printReadyState == true) {

state = 0;

bool firstTime = true;

// Now, since the button is released, enter the printing loop:

while(digitalRead(msPin) == HIGH) { // Runs as long as the switch is not pressed

tempCharNum = ASCIItoInt(receivedChars[state]); // Convert current char

Serial.print("tempCharNum is:");

Serial.println(tempCharNum);

if(firstTime==true) {

// Turn all solenoids off first

for (int temp = 0; temp < 6; temp++) {

digitalWrite(mosfets[temp], LOW);

}

state = 0; // Reset state

firstTime = false;

Serial.println("First loop");

}

if (encoderPos != oldEncPos) {

printBraille();

encoderPos = oldEncPos;

}

delay(150);

showNewData();

}

void recvWithEndMarker() {

static byte ndx = 0;

char endMarker = '\n';

char rc;

while (Serial.available() > 0 && newData == false) {

rc = Serial.read();

if (rc != endMarker) {

receivedChars[ndx] = rc;

ndx++;

if (ndx >= numChars) {

ndx = numChars - 1;

}

else {

receivedChars[ndx] = '\0'; // terminate the string

tone(buzzer, 440, 400);

tone(buzzer, 1000, 400);

tone(buzzer, 440, 150);

tone(buzzer, 1000, 400);

charArraySize = ndx;

ndx = 0;

newData = true;

printReadyState = true;

}

void showNewData() {

if (newData == true) {

Serial.print("This just in ... ");

Serial.println(receivedChars);

newData = false;

}

void printBraille() {

for(int temp=0; temp<6;temp++){

digitalWrite(mosfets[temp], LOW);

}

for(int temp=0; temp<6;temp++){

digitalWrite(mosfets[temp], alphabet[tempCharNum][temp]);

}

void PinA(){

cli(); //stop interrupts happening before we read pin values

reading = PIND & 0xC; // read all eight pin values then strip away all but pinA and pinB's values

if(reading == B00001100 && aFlag) { //check that we have both pins at detent (HIGH) and that we are expecting detent on this pin's rising edge

encoderPos --; //decrement the encoder's position count

if(state>0){

state--; //moves back to the prior letter

}else{state=charArraySize; //goes to end of list if turned to before a

}

bFlag = 0; //reset flags for the next turn

aFlag = 0; //reset flags for the next turn

}

else if (reading == B00000100) bFlag = 1; //signal that we're expecting pinB to signal the transition to detent from free rotation

sei(); //restart interrupts

}

void PinB(){

cli(); //stop interrupts happening before we read pin values

reading = PIND & 0xC; //read all eight pin values then strip away all but pinA and pinB's values

if (reading == B00001100 && bFlag) { //check that we have both pins at detent (HIGH) and that we are expecting detent on this pin's rising edge

encoderPos ++; //increment the encoder's position count

if(state<charArraySize){

state++; //moves onto the next letter

}else{state = 0;

}

bFlag = 0; //reset flags for the next turn

aFlag = 0; //reset flags for the next turn

}

else if (reading == B00001000) aFlag = 1; //signal that we're expecting pinA to signal the transition to detent from free rotation

sei(); //restart interrupts

}

Many thanks to SimonM83 who wrote this rotary encoder instructable. I've used his code in this project.

Testing the Device

After putting everything together and plugging in the power, we can finally test the Braille Vision device. You have to press the microswitch to take a photo and wait until the piezo transducer beeps twice. After the second beep, you can press the microswitch again and begin scrolling through the braille text using the rotary encoder. I've made a youtube video demonstrating the device by translating an image of "Hello World" here.

I hope you've enjoyed reading through this instructable and hopefully it can provide some inspiration for your future Pi-based projects. :)

Downloads

portraitBrailleVideo.mp4