Kinect Laser Harp

This project uses the Xbox Kinect in combination with the program "Processing," to create a low-cost laser harp. The harp plays eight notes, and they can be any sound file in a .wav format, or they can be notes from the sound library included in "Processing."

For the programs needed for the laser harp to run, "Java" must be installed onto your computer. If you already have "Java," then you can proceed to the next step. To install it, go to the website http://www.java.com/en/download/. On this website, click the "Free Java Download" button, and then click the "Agree and Start Free Download" button. Once that has been completed, you can save it to your computer. When you go to open the file, it will give you instructions to complete the install.

The next thing that must be downloaded is "OpenNI." All of the files necessary for this can be downloaded at one time from the website: http://zigfu.com/en/downloads/browserplugin/. Save this to your computer, and follow the instructions that it gives you to complete the download.

To run the Xbox Kinect, you should next download the SDK from: http://www.microsoft.com/en-us/kinectforwindowsdev/start.aspx. This helps in getting your computer to find the Kinect. To download this, click the button labeled "Download Latest SDK," and save it to your computer. Follow the instructions provided, and run it on your computer.

After you have "Java" installed on your computer, it is time to install "Processing." To do this, go to the website: http://processing.org/download/?processing. On this website, you want to download the 32-bit version of "Processing" for Windows. Once you choose to download, save the program to your computer and it will give you instructions to finish the install.

To begin using the Kinect, open "Processing." Once it opens, go to "Import Library," and then choose "Add Library." You need to choose to add the libraries "Open Kinect" and "SimpleOpenNI." Once these are installed, you can begin programming the Kinect.

The easiest way to play notes on the Kinect is to use the program:

import SimpleOpenNI.*;
import ddf.minim.*;
import ddf.minim.ugens.*;

SimpleOpenNI kinect;

int closestValue;
int closestX;
int closestY;

Minim minim;
AudioOutput out;

void setup() {
  size(640, 480);
  kinect = new SimpleOpenNI (this);
  kinect.enableDepth();
  minim = new Minim (this);
  out = minim.getLineOut();
}

void draw() {
  closestValue = 8000;
 
  kinect.update();
 
  int[] depthValues = kinect.depthMap();
 
  for(int y = 0; y < 480; y++) {
    for (int x = 0; x < 640; x++) {
      int i = x + y * 640;
      int currentDepthValue = depthValues[i];
     
      if(currentDepthValue > 0 && currentDepthValue < closestValue) {
      
       closestValue = currentDepthValue;
     
      closestX = x;
      closestY = y;
      }
    }
  }
   
image(kinect.depthImage(), 0, 0);

  fill(255, 0, 0, 155);
  rect(20, 0, 40, 480);
  fill(255, 128, 0, 155);
  rect(100, 0, 40, 480);
  fill(255, 255, 0, 155);
  rect(180, 0, 40, 480);
  fill(0, 255, 0, 155);
  rect(260, 0, 40, 480);
  fill(0, 255, 255, 155);
  rect(340, 0, 40, 480);
  fill(0, 0, 255, 155);
  rect(420, 0, 40, 480);
  fill(128, 0, 255, 155);
  rect(500, 0, 40, 480);
  fill(255, 0, 255, 155);
  rect(580, 0, 40, 480);
 
fill(155);
ellipse(closestX, closestY, 20, 20);

if(closestX < 60 && closestX > 20) {
   out.playNote ("C4");
  }

if(closestX < 140 && closestX > 100) {
    out.playNote ("D4");
  }

if(closestX < 220 && closestX > 180) {
    out.playNote ("E4");
  }

if(closestX < 300 && closestX > 260) {
    out.playNote ("F4");
  }

if(closestX < 380 && closestX > 340) {
    out.playNote ("G4");
  }

if(closestX < 460 && closestX > 420) {
    out.playNote ("A4");
  }

if(closestX < 540 && closestX > 500) {
    out.playNote ("B4");
  }

if(closestX < 620 && closestX > 580) {
    out.playNote ("C5");
  }
}
 
   void stop(){
    minim.stop();
    super.stop();
   }
  
In this program, you can choose the note and the octave by simply typing a new one. For example, if you wanted to raise the last note up one octave, you would replace ("C5") with ("C6").

The other program I used was:

import SimpleOpenNI.*;
import ddf.minim.*;
SimpleOpenNI kinect;

int closestValue;
int closestX;
int closestY;

Minim minim;
AudioSnippet player;

void setup() {
  size(640, 480);
  kinect = new SimpleOpenNI (this);
  kinect.enableDepth();
}

void draw() {
  closestValue = 8000;
 
  kinect.update();
 
  int[] depthValues = kinect.depthMap();
 
  for(int y = 0; y < 480; y++) {
    for (int x = 0; x < 640; x++) {
      int i = x + y * 640;
      int currentDepthValue = depthValues[i];
     
      if(currentDepthValue > 0 && currentDepthValue < closestValue) {
      
       closestValue = currentDepthValue;
     
      closestX = x;
      closestY = y;
      }
    }
  }
   
image(kinect.depthImage(), 0, 0);

  fill(255, 0, 0, 155);
  rect(20, 0, 40, 480);
  fill(255, 128, 0, 155);
  rect(100, 0, 40, 480);
  fill(255, 255, 0, 155);
  rect(180, 0, 40, 480);
  fill(0, 255, 0, 155);
  rect(260, 0, 40, 480);
  fill(0, 255, 255, 155);
  rect(340, 0, 40, 480);
  fill(0, 0, 255, 155);
  rect(420, 0, 40, 480);
  fill(128, 0, 255, 155);
  rect(500, 0, 40, 480);
  fill(255, 0, 255, 155);
  rect(580, 0, 40, 480);
 
fill(155);
ellipse(closestX, closestY, 20, 20);

if(closestX < 60) {
  if(closestX > 20){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound1.wav");
    player.play();
  }
}

if(closestX < 140) {
  if(closestX > 100){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound2.wav");
    player.play();
  }
}

if(closestX < 220) {
  if(closestX > 180){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound3.wav");
    player.play();
  }
}

if(closestX < 300) {
  if(closestX > 260){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound4.wav");
    player.play();
  }
}

if(closestX < 380) {
  if(closestX > 340){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound5.wav");
    player.play();
  }
}

if(closestX < 460) {
  if(closestX > 420){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound6.wav");
    player.play();
  }
}

if(closestX < 540) {
  if(closestX > 500){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound7.wav");
    player.play();
  }
}

if(closestX < 620) {
  if(closestX > 580){
    minim = new Minim(this);
    player = minim.loadSnippet("Sound8.wav");
    player.play();
  }
}
}
 
  void stop(){
    player.close();
    minim.stop();
   
    super.stop();
  }
 
With this program, you can change the sounds to any .wav file. Go under, "Sketch," choose to "Add File," and select the sounds you want. Replace one of the sounds for example, ("Sound8.wav") with the name of new sound that you want, and the program will play the sound you downloaded under that name. With this program, you want to try and keep the sounds as short as possible. If the sounds are more than a few seconds, the program will freeze. This can be fixed under "File" and then "Preferences," up to a certain extent, but the computer really struggles with long clips.

To connect the Kinect to your computer, you need the USB/Power adapter. This allows you to give the Kinect power and have it give information to the computer. If your Kinect was purchased for the use with an Xbox, this will not be included. You can easily find and purchase this online.To run one of the programs, copy the programming portion of it into "Processing," and click the "Run" triangle in the top left. If you have the Kinect plugged in, and all necessary programs are installed, you should see something like the picture in the introduction. As you move, the program shows you a grey ellipse at your closest point to the computer. When this touches a colored bar, that note will play.

The final step of this project is to create a laser array. The lasers need to be placed so that each laser fits in the center of one colored beam. Also, the laser array needs to have something sticking out that fits into the space between beams. Because the harp uses the point closest to the Kinect to determine what note to play, something needs to be in a space so that you do not accidentally play a note. Whenever you pass this object the note will play. Currently, I stand next to a chair while I play, because the array is not yet finished. When I complete this project, more will be added to this Instructable. Until then, I have an air harp, but laser addition will come.

The information I used to create this project was found in the book, "Making Things See" By: Greg Borenstein and in various tutorials throughout the internet. If you want to further develop this project, or create other projects with Kinect, then these are great resources to check out. Also, the Power Adaptor can be found at: http://www.newegg.com/Product/Product.aspx?Item=9SIA1NV0YW9280.