MATLAB Color Recognition

Introduction:

In present scenario home office appliance controlling is common thing. Today for controlling home/offices appliances, machinery in companies/industries uses different types of automation systems like DTMF controlled, GSM controlled and many others. So here this project is made for interfacing MATLAB (Graphic User Interfacing) and Home/office/industries automation using Arduino and MATLAB which can controlled by showing certain symbols / images in front of the webcam. like to switch on the fan you need to show fan symbol in front of the webcam and to switch on the light you need to show light symbol in front of the webcam.

Here MATLAB is interfaced with Arduino through serial communication. Once the MATLAB process the image in the video taken by the webcam, it will send certain serial commands to arduino then,the arduino will turn on / off the respective appliances.

Note: I bought some part of the code from inscrutables, long ago that's why i'm unable recollect their names, but the credit goes to them, as this is not a commercial one,i used their work without proper permission.

Project_files can be found here: (please go through next steps for better understanding)

https://drive.google.com/open?id=0B-Vn6E73-uHxN0xf...

to make This project simple and effective, i followed these steps.

1. i want to detect the red color object then.
2. count the number of objects (red) in the real time video.
3. then i write the code in such way that depending upon the count,mat lab will send the data (please look in the code attachment) to arduino.
4. then write the code for serial read based appliance controlling for arduino.

that's it.

Please look @ the picture carefully, that will give you entire idea

Matlab code:

%Initialise video capturing object
vid = videoinput('winvideo', 1, 'YUY2_640x480');

%SERIAL COMUNICATION%

%Declaring the serial comunication object , to initialise serial %comunication

s=serial('COM9','BAUD',9600); %change COM_ if needed

%open serial port fopen(s);

%VIDEO PROPERTIES SETTNG UP% % Set the properties of the video object

set(vid, 'FramesPerTrigger', Inf); set(vid, 'ReturnedColorspace', 'rgb') vid.FrameGrabInterval = 10;

%start the video aquisition here

start(vid)

% Set a loop that stop after 100 frames of aquisition %for i=1:100

while(vid.FramesAcquired<=20000)

% Get the snapshot of the current frame

IMRED = getsnapshot(vid);

% Now to track red objects in real time%

% we have to subtractED the red component % from the grayscale image to extract the red components in the image.

diff_im = imsubtract(IMRED(:,:,1), rgb2gray(IMRED));

gr=graythresh(diff_im);

%Median filter IS USED to filter out noise

diff_im = medfilt2(diff_im, [3 3]);

% Convert the resulting grayscale image into a binary image.

diff_im = im2bw(diff_im,0.18); % Remove all those pixels less than 300px

diff_im = bwareaopen(diff_im,300); % Label all the connected components in the image

%and also count the nuber of red objects in frame

[bw bw1] = bwlabel(diff_im, 8);

% condition if one or more than one red object is present in frame,then %send value '100' else send 101

if bw1=1

fprintf(s,100); elseif bw1==0 fprintf(s,101);

if bw1=2
fprintf(s,110); elseif bw1==0 fprintf(s,111);

if bw1=3
fprintf(s,120); elseif bw1==0 fprintf(s,121);

% Here we do the image blob analysis.

% We get a set of properties for each labeled region.

stats = regionprops(bw, 'BoundingBox', 'Centroid');

% Display the image

imshow( IMRED )

hold on

%This is a loop to bound the red objects in a rectangular box.

for object = 1:length(stats)

bb = stats(object).BoundingBox;

bc = stats(object).Centroid;

rectangle('Position',bb,'EdgeColor','r','LineWidth',2)

plot(bc(1),bc(2), '-m+')

a=text(bc(1)+15,bc(2), strcat('X: ', num2str(round(bc(1))), ' Y: ', num2str(round(bc(2))))); set(a, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'yellow');

end

hold off

end

end

% Both the loops end her. % Stop the video aquisition.

stop(vid);

% Flush all the image data stored in the memory buffer.

flushdata(vid);

%Close the serial port ,so that further communication could be establihed %again.

fclose(s); % Clear all variables clear all

sprintf('%s','Congratulations')

Arduino code:

const int rpin=2;
const int gpin=5; const int bpin=8; int recValue;

void setup()

{

Serial.begin(9600);

pinMode(2, OUTPUT); pinMode(5, OUTPUT); pinMode(8, OUTPUT);

}

void loop()

{

if(Serial.available()>0)

{

recValue=Serial.read();

if (recValue == 100) // If use will send value 100 from MATLAB then r LED will turn ON

{ digitalWrite(rpin, HIGH); }

if(recValue == 101) // If use will send value 101 from MATLAB then r LED will turn OFF

{ digitalWrite(rpin, LOW); }

if (recValue == 110) // If use will send value 110 from MATLAB then gLED will turn ON

{ digitalWrite(gpin, HIGH); } if(recValue == 111) // If use will send value 111 from MATLAB then gLED will turn OFF

{ digitalWrite(gpin, LOW); } if (recValue == 120) // If use will send value 120 from MATLAB then bLED will turn ON

{ digitalWrite(bpin, HIGH); } if(recValue == 121) // If use will send value 121 from MATLAB then bLED will turn OFF

{ digitalWrite(bpin, LOW); } }

}

to make This project simple and effective, i followed these steps.

• i want to detect the red,green,blue colors object then.
• count the number of objects (red/green/blue) in the real time video.
• then i write the code in such way that depending upon the respective count >=1, mat lab will send the data (please look in the code attachment) to arduino
• .then write the code for serial read based appliance controlling for arduino. that's it.

Please look @ the picture carefully, that will give you entire idea

Matlab code:

redThresh = 0.20; % Threshold for red detection
greenThresh = 0.10; % Threshold for green detection

blueThresh = 0.15; % Threshold for blue detection

s=serial('COM8','BAUD',9600); %open serial port fopen(s);

vidDevice = imaq.VideoDevice('winvideo', 1, 'YUY2_640x480', ... % Acquire input video stream

'ROI', [1 1 640 480], ... 'ReturnedColorSpace', 'rgb');

vidInfo = imaqhwinfo(vidDevice); % Acquire input video property

hblob = vision.BlobAnalysis('AreaOutputPort', false, ... % Set blob analysis handling 'CentroidOutputPort', true, ... 'BoundingBoxOutputPort', true', ... 'MinimumBlobArea', 600, ... 'MaximumBlobArea', 3000, ... 'MaximumCount', 10);

hshapeinsBox = vision.ShapeInserter('BorderColorSource', 'Input port', ... % Set box handling 'Fill', true, ... 'FillColorSource', 'Input port', ... 'Opacity', 0.4);

htextinsRed = vision.TextInserter('Text', 'Red : %2d', ... % Set text for number of blobs 'Location', [5 2], ... 'Color', [1 0 0], ... // red color 'Font', 'Courier New', ... 'FontSize', 14);

htextinsGreen = vision.TextInserter('Text', 'Green : %2d', ... % Set text for number of blobs 'Location', [5 18], ... 'Color', [0 1 0], ... // green color 'Font', 'Courier New', ... 'FontSize', 14);

htextinsBlue = vision.TextInserter('Text', 'Blue : %2d', ... % Set text for number of blobs 'Location', [5 34], ... 'Color', [0 0 1], ... // blue color 'Font', 'Courier New', ... 'FontSize', 14);

htextinsCent = vision.TextInserter('Text', '+ X:%4d, Y:%4d', ... % set text for centroid 'LocationSource', 'Input port', ... 'Color', [1 1 0], ... // yellow color 'Font', 'Courier New', ... 'FontSize', 15);

hVideoIn = vision.VideoPlayer('Name', 'Final Video', ... % Output video player 'Position', [100 100 vidInfo.MaxWidth+20 vidInfo.MaxHeight+30]); nFrame = 0; % Frame number initialization

%% Processing Loop

while(nFrame < 3000)

rgbFrame = step(vidDevice); % Acquire single frame

rgbFrame = flipdim(rgbFrame,2); % obtain the mirror image for displaying

diffFrameRed = imsubtract(rgbFrame(:,:,1),

rgb2gray(rgbFrame)); % Get red component of the image

diffFrameRed = medfilt2(diffFrameRed, [3 3]); % Filter out the noise by using median filter

binFrameRed = im2bw(diffFrameRed, redThresh); % Convert the image into binary image with the red objects as white

%binFrameRed = bwareaopen(binFrameRed,300);

[bw bw1] = bwlabel(binFrameRed, 8);

if bw1>=1

fprintf(s,101);

else

fprintf(s,100);

end

diffFrameGreen = imsubtract(rgbFrame(:,:,2), rgb2gray(rgbFrame)); % Get green component of the image diffFrameGreen = medfilt2(diffFrameGreen, [3 3]); % Filter out the noise by using median filter

binFrameGreen = im2bw(diffFrameGreen, greenThresh); % Convert the image into binary image with the green objects as white

%binFrameRed = bwareaopen(binFrameGreen,300);

[bw bw1] = bwlabel(binFrameGreen, 8);

if bw1>=1

fprintf(s,102);

else

fprintf(s,103);

end

diffFrameBlue = imsubtract(rgbFrame(:,:,3), rgb2gray(rgbFrame)); % Get blue component of the image diffFrameBlue = medfilt2(diffFrameBlue, [3 3]); % Filter out the noise by using median filter

binFrameBlue = im2bw(diffFrameBlue, blueThresh); % Convert the image into binary image with the blue objects as white

%binFrameRed = bwareaopen(binFrameBlue,300);

[bw bw1] = bwlabel(binFrameBlue, 8);

if bw1>=1

fprintf(s,104);

else

fprintf(s,105);

end

[centroidRed, bboxRed] = step(hblob, binFrameRed); % Get the centroids and bounding boxes of the red blobs centroidRed = uint16(centroidRed); % Convert the centroids into Integer for further steps

[centroidGreen, bboxGreen] = step(hblob, binFrameGreen); % Get the centroids and bounding boxes of the green blobs centroidGreen = uint16(centroidGreen); % Convert the centroids into Integer for further steps [centroidBlue, bboxBlue] = step(hblob, binFrameBlue); % Get the centroids and bounding boxes of the blue blobs centroidBlue = uint16(centroidBlue); % Convert the centroids into Integer for further steps rgbFrame(1:50,1:90,:) = 0; % put a black region on the output stream

vidIn = step(hshapeinsBox, rgbFrame, bboxRed, single([1 0 0])); % Instert the red box

vidIn = step(hshapeinsBox, vidIn, bboxGreen, single([0 1 0])); % Instert the green box

vidIn = step(hshapeinsBox, vidIn, bboxBlue, single([0 0 1])); % Instert the blue box

for object = 1:1:length(bboxRed(:,1)) % Write the corresponding centroids for red

centXRed = centroidRed(object,1);

centYRed = centroidRed(object,2);

vidIn = step(htextinsCent, vidIn, [centXRed centYRed], [centXRed-6 centYRed-9]);

end

for object = 1:1:length(bboxGreen(:,1)) % Write the corresponding centroids for green

centXGreen = centroidGreen(object,1);

centYGreen = centroidGreen(object,2);

vidIn = step(htextinsCent, vidIn, [centXGreen centYGreen], [centXGreen-6 centYGreen-9]);

end

for object = 1:1:length(bboxBlue(:,1)) % Write the corresponding centroids for blue

centXBlue = centroidBlue(object,1); centYBlue = centroidBlue(object,2);

vidIn = step(htextinsCent, vidIn, [centXBlue centYBlue], [centXBlue-6 centYBlue-9]);

end

vidIn = step(htextinsRed, vidIn, uint8(length(bboxRed(:,1)))); % Count the number of red blobs

vidIn = step(htextinsGreen, vidIn, uint8(length(bboxGreen(:,1)))); % Count the number of green blobs

vidIn = step(htextinsBlue, vidIn, uint8(length(bboxBlue(:,1)))); % Count the number of blue blobs

step(hVideoIn, vidIn); % Output video stream

nFrame = nFrame+1;

end

%% Clearing Memory release(hVideoIn); % Release all memory and buffer used

release(vidDevice);

clear all;

clc;

Arduino code:

const int led1pin=2;
const int led2pin=3; const int led3pin=4; int recValue;

void setup()

{

Serial.begin(9600);

pinMode(2, OUTPUT); pinMode(3, OUTPUT); pinMode(4, OUTPUT); pinMode(5, OUTPUT);

}

void loop()

{

if(Serial.available()>0)

{

recValue=Serial.read();

if (recValue == 100) // If use will send value 100 from MATLAB then LED will turn ON

{ digitalWrite(led1pin, HIGH); }

if(recValue == 101) // If use will send value 101 from MATLAB then LED will turn OFF

{ digitalWrite(led1pin, LOW); } if (recValue == 102) // If use will send value 100 from MATLAB then LED will turn ON

{ digitalWrite(led2pin, HIGH); }

if(recValue == 103) // If use will send value 103 from MATLAB then LED will turn OFF

{ digitalWrite(led2pin, LOW); } if (recValue == 104) // If use will send value 100 from MATLAB then LED will turn ON

{ digitalWrite(led3pin, HIGH); }

if(recValue == 105) // If use will send value 105 from MATLAB then LED will turn OFF

{ digitalWrite(led3pin, LOW); }

}

}